High-level Design of Sharded Counters
Discover why high-volume features like social media likes require distributed counting systems. Learn to design sharded counters that distribute write requests across parallel computational units to eliminate contention and improve scalability. Outline the necessary APIs for creating, updating, and reading these high-performance counters.
High-level solution sketch
Handling millions of likes on a single tweet can create contention on a shared counter. If each like generates a write request to the same counter, the updates are serialized at that resource. This serialization increases write latency and limits overall throughput.
Real-time applications require low latency. To achieve this, we must eliminate the bottleneck caused by concurrent writes to a single data point.
The following illustration demonstrates this problem:
A single counter cannot handle millions of concurrent writes. The solution is a sharded counter (or distributed counter). This approach splits the counter into multiple shards, each running in parallel on a different node. By balancing write requests across these shards, we reduce contention and improve performance. ... ...