Client-side Load Balancer for Twitter
Learn why Twitter’s microservices often use client-side load balancing instead of relying only on centralized load balancers. Uncover techniques used at Twitter, including the Power of Two Choices (P2C) algorithm for request distribution. Understand deterministic aperture and ring-based coordination, which improve fairness, scalability, and routing stability under high traffic.
Introduction
The previous lesson modeled Twitter with a centralized load balancer. Although functional, this design does not scale efficiently for Twitter’s traffic patterns. Twitter operates many heterogeneous services across large instance fleets, which can overwhelm centralized load balancers and create throughput bottlenecks. Understanding the solution requires examining the evolution of Twitter’s architecture.
Twitter’s design history
Twitter began as a monolithic Ruby on Rails application with a MySQL database. As the platform scaled, the database was sharded, but the monolithic application remained problematic:
Codebase contention: A large number of developers working on the same codebase made it difficult to update individual services.
Cascading failures: An upgrade in one service could break others.
High costs: Single machines performing numerous services led ...