Preface We’ve heard of Log-Structured Merge-Tree (LSMTree) for many times recently, when many high scalable NoSQLs are gaining popularity such as Cassandra, HBase, LevelDB, Elasticsearch… We may have a very brief idea on how they work under the hood theoretically, but we haven’t touch that in depth. …

In the very beginning, I was wondering how a webserver works, how mighty Apache — extremely dominating in the market at that time — was struggling with high traffic environments, how Nginx’s event-driven architecture beautifully solved C10K problem, and long after, how NodeJS got the community hype. …

We’ve probably heard of, or even used Redis quite frequently as part of our infrastructure. Redis is everywhere, which can be a caching layer, a message queue, or even a memory database, whatever it is, performance is always the main objective of using Redis. In Coccoc infrastructure, Redis has being…

Foreword In the previous blogpost, we’ve discussed the neediness of consensus protocol in the distributed computing era and the most commonly known 2-phase-commit algorithm. …

Foreword You may have heard of Transaction — which is very important feature to maintain atomicity of the database. For those who dont know, atomicity guarantees updates to the database either all occur, or nothing occurs. On a single node database, transaction is usually implemented by the storage engine, using a…

Why Zero-copy? What’s happening under the hood when the OS is copying a file / transfering a file to another host? For our naked eyes the process can be simple, OS first reads content of the file, then writes it to another file, then it’s done! However, things become complicated when we…

We have several Kafka clusters of growing importance at CocCoc. As usage scales up, we would like to get a handle on the dataflow components. We decided to enforce producer/consumer ACLs on them to ensure data provenance, using Kafka built-in Authorizer which stores settings in Zookeeper. Starting points Kafka cluster with PLAINTEXT…

Coccoc uses KVM (Kernel-based Virtual Machine) virtualization infrastructure as a private cloud solution, and improving its performance and stability is always a challenging yet exciting task. In this blogpost, we will go through a brief description of cpu flags, and see how a little tweak on CPU flags can improve…

Preface For a very long time, the infrastructure at Coccoc only supports traditional services running on physical hosts. Recently on the rise of microservices, the trendy container orchestration technology K8s has been adapted as a big part of the grand design. Therefore, the infrastructure also needs a huge shift to not…