Advanced Details of Load Balancers
Discover how load balancers are deployed in modern data centers using a multi-tier hierarchy. Analyze the differences between Layer 4 and Layer 7 LBs, and evaluate static vs. dynamic algorithms for optimal request distribution. Understand state management to build resilient System Design solutions.
This lesson covers common load balancing algorithms and how load balancers (LBs) are organized hierarchically to distribute traffic across system tiers.
Algorithms of load balancers
Load balancers use specific algorithms to distribute client requests, which include:
Round robin scheduling: Requests are forwarded to servers sequentially in a repeating loop.
Weighted round robin: Useful when servers have varying capacities. Each node is assigned a weight; servers with higher weights receive a larger proportion of requests.
Least connections: Assigns new requests to the server with the fewest active connections. This is effective when request processing times vary significantly, preventing servers with long-running tasks from becoming overwhelmed.
Least response time: Directs traffic to the server with the lowest active response time, optimizing for latency. It is useful in performance-sensitive services.
IP hash: Hashes the client’s IP address to map them to a specific server. This is often used to maintain session stickiness.
URL hash: Distributes requests based on the URL. This routes traffic to specific server clusters handling distinct services (e.g., separating video serving from text).
Other variations exist, such as randomized or weighted least connections algorithms.
Static vs. dynamic algorithms
Algorithms are categorized based on whether they consider the machine’s state:
Static algorithms (e.g., Round robin) distribute tasks based on fixed configurations. They are simple and low-overhead, but do not account for real-time server health or load.
Dynamic algorithms (e.g., Least connections) monitor the current state of servers. While they add complexity and communication overhead, they improve reliability by routing around overloaded or unhealthy servers.
Dynamic algorithms require load balancers to exchange information, making them modular but complex. However, they provide better forwarding decisions by actively monitoring server health.
Note: In practice, dynamic algorithms provide superior results because they maintain the state of serving hosts, justifying the added complexity.
Stateful vs. stateless LBs
Load balancers may need to handle session persistence (state) between clients and hosting servers.
If ...