System Design: Sharded Counters

Problem statement

Real-time applications like Facebook, Twitter, and YouTube have high user traffic. Users interact with the applications and perform multiple operations (view, like, comment, etc.) depending on the application’s structure. For instance, an image is posted on a Facebook page that has millions of followers, and the post likes rapidly increase after each millisecond. Here, counting the likes might be easy for this single image, but what will we do when thousands of such images or videos are uploaded simultaneously by many celebrities, each with millions of followers. This problem is known as the heavy hitter problem. The above scenario shows how a simple counting operation becomes challenging to manage with precision and performance. The following figure shows which of YouTube’s videos were viewed by millions of users in a 24 hours span as of August 2021.