MyDistributed.Systems

When building networked applications in Java, it helps to understand how TCP, HTTP, Netty, and gRPC fit together. This article walks through the layers of the stack, how request/response boundaries work, and how frameworks like Netty and gRPC help you build fast systems without wrestling with raw bytes. Transport Layer At its core, TCP (Transmission Control Protocol) is a reliable, ordered stream of bytes. Thinking about a TCP connection is easy if you imagine it as two pipes: one from the client to the server and one from the server back to the client. Each pipe just carries bytes. TCP does NOT know about request or response boundaries. It delivers bytes in order, and it takes care of retransmissions, acknowledgements, and flow control so you do not have to. Following our pipe analogy, when you read from a TCP connection, imagine placing a bucket under the pipe; bytes drip in and you accumulate them until you hav...

Ten years after the public availability of DynamoDB in 2012, and fifteen years after the publication of the original Dynamo paper in 2007, the DynamoDB team published a paper in Usenix ATC 2022 on how DynamoDB has evolved over its ten years of operation. The paper covers several aspects of DynamoDB such as adaptation to customer access patterns, smarter capacity allocation, durability, correctness, and availability. In this post, I share some of points that I found interesting in the paper. 

Dual data structures are concurrent data structures that not only hold data, but also keep track of read requests using reservations. By holding both data and anti-data, dual data structures achieve better performance and fairness compared with other blocking and non-blocking concurrent data structures.

This is the second part of a two-part post on eventual consistency and the problem of conflict resolution. In the  first part , we focused on the definition of the eventual consistency, its sufficient conditions, and conflict resolution using physical and logical clocks. We saw each method has its problems. In this part, we will see how we can solve these issues and provide eventual consistency with better conflict resolution. Listen to the Audio Blog Conflict Resolution using Vector Clocks The good thing about physical clock timestamps is that they carry information related to the real-time order of events, e.g., if we write v2, one minute after writing v1, the physical timestamp of v2 will be most likely larger than that of v1, assuming we have reasonable clock synchronization in place. However, physical clocks are not 100% accurate. Thus, due to clock synchronization errors, the timestamps may not reflect ...