Client-side Load Balancer for Twitter

Introduction

In the previous lesson, we conceived the design of Twitter using a dedicated load balancer. All end-users requests go through a dedicated load balancing layer into different services. Although this method works, and we employed it in other designs, it may not be the optimal choice for Twitter. It is because Twitter offers a variety of services at a large scale using numerous instances and dedicated load-balancers are not a suitable choice for such systems. To understand the concept better, let’s understand the history of Twitter design.

The design history of Twitter

The initial design of Twitter included a monolithic (Ruby on Rails) application with a MySQL database. As Twitter scaled, the number of services increased and the MySQL database was sharded. With a monolithic application, such a design is a disaster because of the following reasons:

• A large number of developers work on the same codebase which makes it complex to update individual services
• The upgrade process of one service may lead to the breaking of another
• Hardware costs grow because a single machine performs numerous services
• Recovery from failures is both time-consuming and complex

With the way Twitter evolved, the only way out was many microservices where each service can be served through hundreds or thousands of instances.

Client-side load balancing

In a client-side load balancing technique, there is no dedicated intermediate load balancing infrastructure between any two services with a large number of instances. A node requesting a set of nodes for a service has a built-in load balancer. We refer to the requesting node/service as a client. The client can use a variety of techniques to select a suitable instance for requesting the service. The illustration below depicts the concept of client-side load balancing. Every new arriving service/instance will register itself with a service registry so that other services are aware of its existence