Lock in Golang

Golang recommends using channel to do communicating between goroutines. The idea of channel comes from data flow. But in some scenarios, using channel is too heavy to implement, mutex is a better solution. There are two kinds of locks in Golang, sync.Mutex,sync.RWMutex. The difference between Mutex and RWMutex is that RWMutex is a reader/writer mutual exclusion lock, the lock can be held by an arbitrary number of readers or a single writer, readers don’t have to wait for each other, They only have to wait for writers holding the lock.

Real-time data warehouse examples

The concept of data warehouse may be traced back to last century. with the continuous growth of big data and developments of Hadoop ecosystem, offline data warehouse based on Hive/HDFS architect can rise. And recently years, Storm/Spark(Steaming)/Flink etc… real-time frameworks go up and have a rapid development. Every company need a real-time data solution in their system. In this article, we will talk about some typical real-time data architect in Chinese internet companies like Meituan,Netease and OPPO; selections of their storage and computing engines, also with layers division may inspire us in some point.

Memory Management in Golang

Golang’s memory management model is based upon tcmalloc pattern. TCMalloc is Google’s customized implementation of c’s malloc and c++’s operator new used for memory allocation within c or c++ code. We know that malloc invoked will trigger threads switch into kernel space, it’s a big consumption. So TCMalloc provide thread cache capability to enhance the performance. In golang, the local cache is not specify to thread, but P, processors in GMP schedule Model.

How does GMP scheduler work

As we know that There is a runtime when Golang running. The runtime perform the scheduled tasks(goroutine) in user space rather than kernel, so it’s more lightweight. It do a better tradeoff between system resource usage and performance, especially in IO tasks. In this article, I’ll show you Golang scheduler’s history, Goroutine scheduler GMP’s design pattern, and some cases how does GMP handle.

Why we need a Scheduler

As we know that There is a runtime when Golang running. The runtime perform the scheduled tasks(goroutine) in user space rather than kernel, so it’s more lightweight. It do a better tradeoff between system resource usage and performance, especially in IO tasks. In this article, I’ll show you Golang scheduler’s history, Goroutine scheduler GMP’s design pattern, and some cases how does GMP handle.

GREP and regex basic

GREP is global regular expression print. It may the most popular tool to find patterns in file and it is always a built-in command in *nix system. We can see it from name that grep can match string from input data with regular expression. In this article, I’ll show you how to use grep and some normal regular expression meanings in daily scenario.

Overall usage of grep

Testing in Golang

Golang has a default command go test to run test code like junit. go test will think of the package as a basic unit. Of course you can specify the source files you run, that filename endswith “_test.go”. And testing function always signed like func TestXXX(t testing.T).

幂等与全局ID

什么是幂等?我们这里所说的幂等是指非主观因素造成了多次相同的请求,而做幂等设计要求这些相同的请求无论是做一次还是多次不产生副作用。即 f...f(f(x))=f(x)。产生多次请求原因有很多比如前端没有做防抖的多次点击提交,MQ消息投递失败重试等。

Java容器类

Java提供的容器类如List、Set、Map是语言的基础,熟练掌握是必备技能,本文就Java中容器相关的坑与知识做一个总结整理。本文涉及的代码摘自OpenJDK jdk9-b94,部分逻辑在不同版本的实现有所差异。

BIO、NIO与AIO

IO是操作系统提供的基础功能,之前介绍了网络Server模型的演进就是基于IO的改进。在语言层面,不同语言对操作系统的IO能力进行了不同的封装与接口的统一,如Java的NIO在linux平台使用的epoll而windows平台使用的iocp,这也是Java作为高级语言层面对操作系统功能统一封装的一个体现。本文结合Java具体代码了解一下什么是BIO、NIO与AIO。