Cover the basics and learn how to design and develop distributed systems.

Become a Distributed Systems Professional

Gone are the days when monolithic applications were the norm. Most applications we use today are too complex for that type of construction. Instead, most applications that appear as a single unit are actually built as a collection of inter-operating but independent computational components. Developers are expected to have mastery over a programming language or two. Moreover, they should also know the basics of distributed systems.

In this module, we’ll learn what distributed systems are, and learn about the challenges unique to distributed systems. We’ll cover popular architectural patterns for achieving targets like replication and sharding. We’ll complete the module with a study of two popular distributed systems: Apache Spark and Apache Druid. With the knowledge gained through this module, we’ll be able to design and develop distributed systems. We’ll also develop an understanding of optimal design and architectural choices based on the application requirements.

Introduction to Distributed Systems for Dummies

# How do LBs balance the load?

There are several load balancing algorithms that are commonly used in the industry. Let's discuss the algorithms in two categories: application-layer algorithms and network-layer algorithms.

## Application-layer algorithms

In application-layer load balancing, the load balancer has access to the data of the request. It can take decisions based on the request headers as well as the request body. Let's discuss popular application-layer load balancing algorithms. 

### Hashing

A common application-layer algorithm is **hashing**. The LB can hash a set of predefined attributes and generate a hash value. The hash value is then mapped to one of the server nodes. Let's give a simple example.

  Assume that the request body from a client contains a `user_ID`. Upon receiving the request, the LB can do the following steps

1. Hash the `user_ID` using any pre-configured hash function. Say for `user_ID = abc123`, the hash function returns a value `981723123`.
2. Take the returned value and map it to a server node. This can be done by using the modulo operation `981723123 % 5` which equals `3`. Here, we are taking a modulo by `5` as this is the count of nodes in the system.
3. Route the request to node `3`.



### Endpoint evaluation

In this mechanism, the endpoint of the request is considered and routed accordingly. For example,  let’s say we have two incoming requests. One hits the endpoint `/v1/app/photos` and the other hits `/v1/app/videos`. In such a setup, you might have two sets of servers for serving photos and videos respectively. The load balancer will evaluate the endpoints and route the request to the respective servers.



## Network-layer algorithms

In network-layer load balancing, the load balancer does not access the data of the request. It can see the source and destination of the request but not what is there in the request. There are a couple of common algorithms based on the network layer.

### Random selection

This is a very simple algorithm to send the request to any one of the server nodes randomly. The LB calculates a probability for choosing server nodes and decides accordingly. Sometimes a sophisticated random number generation is used to choose a node.

### Round robin 

In this algorithm, the requests are routed to servers in a round-robin manner. Say you have three servers numbered $1$, $2$ and $3$, and 5 requests.

  Upon receiving the first request, it is routed to server $1$. The next request will go to server $2$. Then the next one to $3$. $4th$ and $5th$ request will go to server $1$ and $2$.

  The LB will keep a state of the server and route the request accordingly.

### Least connection

In this algorithm, LBs keep track of how many persistent connections there are in the server nodes and choose the one with the least number of connections. In systems where persistent connections are common (think of chat application backends), least connection can be a beneficial choice for load balancing.

### IP hashing

In this algorithm, the source IP is hashed using a good and fast hash function. Based on the hashed value, the request is routed to a specific server.

### Least pending requests

In this algorithm, the load balancer monitors the count of the pending requests in a server and routes the incoming request to the server with the least pending requests. This essentially means that the load balancer is evaluating how long a request is taking to be resolved in a server. If requests are taking too long, the server will have more pending requests. As a result, the load balancer will try to optimize requests by sending new incoming requests to the less busy servers.

---

> The term **network-layer** and **application-layer** are from the infamous OSI model in computer networking. If you are unfamiliar with this concept, please refer to your networking textbook. There are also some resources available at the end of this course.

# Which algorithm should you choose?

This is very system-specific. One algorithm working for a system does not necessarily mean it will be similarly suitable for other systems. For instance:

- You first need to decide which type of load balancing is needed for your system: network-layer or application-layer. Based on that, you can choose a particular algorithm.
- If a system has very short request-response patterns, then round-robin or random both can be suitable algorithms.
- If the system does not have a uniform pattern (for example, some requests may require a much longer time to respond to) then the round-robin or random algorithm won’t be suitable at all. This is because one of the servers could be stretched thin with a few requests, and these algorithms would continuously add more load on to them. In such systems, the least connection or the least response time algorithm makes more sense.
- On the other hand, even hashing with a good hash function could create issues if the particular client puts more load on the server. For example, assume a LB uses IP hashing to balance the load. If clients from an IP send more requests, then the corresponding server would be more loaded.

# Key takeaways
- It's crucial to carefully choose the load balancing type for your system.
- There are multiple options to choose from for load balancing algorithms. As a developer, you need to understand the algorithms.
- You must carefully evaluate your system requirements to choose a suitable algorithm for load balancing.

Learn some common algorithms for load balancing.

Load Balancing Algorithms

How do LBs balance the load?