Introduction to Load Balancers

What is load balancing?

Typical data centers receive millions of requests per second. To handle this volume, thousands of servers work together to share the load. A load balancer (LB) distributes incoming client requests across a pool of available servers. Its primary job is to divide traffic fairly to prevent any single server from crashing due to overload.

Why do we need load balancers?

The load balancer is the first point of contact inside a data center after the firewall. While low-traffic services may not require one, load balancers provide critical capabilities as traffic increases:

• Scalability: You can add servers to increase capacity seamlessly. Load balancers make upscaling or downscaling transparent to end users.

• Availability: If a server fails, the system remains online. The load balancer detects faults and reroutes traffic to healthy servers.

• Performance: Load balancers can forward requests to servers with the lowest load, improving response times and resource utilization.

Here’s an abstract depiction of how load balancers work:

Load balancers distribute requests using configured algorithms, such as:

• Round robin

• Weighted round robin

• Least response time

• Least connections

LBs ensure reliability by continuously monitoring server health and directing traffic only to servers that can respond efficiently. We will explore these algorithms in the Advanced Details of Load Balancers lesson.

Placing load balancers

Load balancers are commonly deployed between clients and backend servers, but they can also operate between internal service layers. In a typical three-tier architecture, load balancers are placed at the following points:

• Between end users and web servers (or the application gateway).

• Between web servers and application servers.

• Between application servers and database servers.

In reality, load balancers can be used between any two services with multiple instances.

Services offered by load balancers

Beyond scalability and availability, load balancers offer several key services:

• Health checking: LBs use the heartbeat protocolThe heartbeat protocol is a way of identifying failures in distributed systems. Using this protocol, every node in a cluster periodically reports its health to a monitoring service. to monitor the health and reliability of servers.

• TLS terminationTLS termination, also called TLS/SSL offloading, is the establishment of secure communication channels between clients and servers through encryption/decryption of data.: LBs handle TLS termination, reducing the processing burden on backend servers.

• Predictive analytics: LBs analyze traffic patterns to predict usage trends.

• Reduced human intervention: Automated failure handling reduces the need for manual system administration.

• Service discovery: LBs route requests to the appropriate hosting servers by querying the service registryService registry is a repository of the (micro)services and the instances available against each service..

• Security: LBs mitigate attacks like denial-of-service (DoS)The DoS is an attack where a client floods the server with traffic to exhaust the server’s resources (processing and/or memory) such that it is unable to process a legitimate user’s request. at layers 3, 4, and 7 of the OSI modelThe open systems interconnection (OSI) model is a conceptual framework that divides the problem of connecting any two machines into seven different layers..

Overall, load balancers provide flexibilityAdd or remove machines transparently on the fly., reliabilityBuggy hosts can be removed through health monitoring which makes the system reliable., redundancyMultiple paths leading to the same destination or failed server’s load is rerouted to the failover machine., and efficiencyDivide load evenly on all machines to use them effectively from the point of view of the service provider. to the System Design.

In the next lessons, we’ll look at how load balancers operate in complex systems and how to choose the right type for different use cases.