Advanced Details of Load Balancers

In this lesson, we will understand what local Load Balancers (LBs) are and some of the well-known algorithms used within these types of load balancers. We will also understand how load balancers are connected to form a hierarchy sharing work across different tiers of LBs.

What are local load balancers?

Local load balancers reside within a data center, behaving like a reverse proxyReverse proxy is a component at the edge of the server-side network that sits between the server and the outside world. It can provide features like privacy, security, caching, etc. and do its best effort to divide incoming requests among the pool of available servers. Incoming clients’ requests seamlessly connect to the LB that uses a Virtual IP address ( VIPVIP).

Algorithms of load balancers

Load balancers distribute client requests according to an algorithm. Some well-known algorithms are given below:

• Round-robin scheduling: Each request is forwarded to a server in the pool in a repeating sequential manner.
• Weighted round-robin: If some servers have a higher capability of serving clients’ requests, then it is preferred to use a weighted round-robin algorithm. In a weighted Round-robin algorithm, each node is assigned a weight. LBs forward clients’ requests according to the weight of the node. The higher the weight, the higher the number of assignments.
• Least connections: In certain cases, even if all the servers have the same capacity to serve clients, uneven load on certain servers is still a possibility. For example, some clients may have a request that requires longer to serve. Or some clients may have subsequent requests on the same connection. In that case, we can make use of algorithms like least connections where newer arriving requests are assigned to servers with fewer existing connections. LBs will keep a state of the number and mapping of existing connections in such a scenario. We will discuss more about state maintenance later in the lesson.
• Least response time: In performance-sensitive services, algorithms like least response time will be required, ensuring that the server with the least response time is requested to serve the clients.
• IP hash: Some applications will provide a different level of service to users based on their IP addresses. In that case, hashing the IP address is performed to assign users’ requests to servers.
• URL hash: It may be possible that some services within the application are provided by specific servers only. In that case, a client requesting service from a URL will be assigned to a certain cluster/set of servers. URL hashing algorithm will be used in those scenarios.

There are still other algorithms like randomized or weighted least connections algorithms.

Static vs. dynamic algorithms

Algorithms can be static or dynamic depending on the machine’s state. Let’s look at each of the categories individually:

• Static algorithms: These algorithms don’t consider the changing state of the servers. Therefore, task assignment is carried out based on existing knowledge about the server’s configuration. Naturally, these algorithms are not complex and get implemented in a single router or commodity machine where all the requests arrive.
• Dynamic algorithms: If an algorithm takes into account the current/recent state of the servers, such algorithms are referred to as dynamic algorithms. Dynamic algorithms maintain state by communicating with the server which adds a communication overhead. State maintenance makes the design of the algorithm much more complicated. Dynamic algorithms require different load balancing servers to communicate with each other to exchange information. Therefore, dynamic algorithms can be modular because no single entity will do decision-making. Although this adds complexity to dynamic algorithms, it results in improved forwarding decisions. Finally, dynamic algorithms monitor the health of the servers and forwards requests to active servers only.

In practice, dynamic algorithms provide far better results because they maintain a state of serving hosts and are, therefore, worth the effort and complexity.