A practitioner's guide to building scalable systems

Building Scalable Systems: Distributed Systems basics to System Design Interviews

We want to give you the tools to compare multiple approaches and pick the one that works for your scenario.

Grokking the Distributed and Scalable Systems

# Routing in Load Balancers
We have seen that Load Balancers (both L4 and L7) extract data from the request and then uses that data to decide which server should get this traffic. Load Balancers deploy different techniques and use the data from the application as input to those techniques/algorithms that determine where the current request would be sent.

The goal is these algorithms is to find the server that can process the current request with the least amount of delay. Here are the most common algorithms:



## 1. Round Robin
Load Balancer has a list of the servers and it sends the first request to the first server in the list, second to the second server and so on. When it reaches the end of the list, it starts over. 

Round Robin Load Balancing is the fastest/easiest way to balance traffic. It would work well if almost all requests take the same time to process on any server. However, In practice, the processing time for the requests can vary significantly and hence it can be suboptimal in those cases.

## 2. Least Number of Connections
Least Number of connections is usually the default Load Balancer algorithm. It looks at the active connections on each server (_requests in flight_) and then sends the request to the server with the least number of open connections. 

It works in most cases as the _chances_ of contention increases with more pending requests and hence a server which has fewer pending requests is most likely to get to it the request fastest.

_In the real world, we go to McDonald and stand in the queue with the least number of customers. This usually works but if the person ahead of you is ordering for 20 people, it would be less efficient for you_

## 3. Least Response Time
Least Response time refers to the algorithm where Load Balancer looks at the last few responses of each server and forwards the request to the server that has the lowest response time in the past few minutes. 

There is again no guarantee that the server with the least response time will continue to respond quickly. Also, there's a risk if Load Balancer can over-burden the server by sending too many requests to that one server.

# There is no perfect Algorithm
Like so many other things, there is no perfect algorithm for load balancing and you would have to configure & tweak the algorithms based on your use case and traffic pattern.



What and Why

Node in a Distributed System

Performance and Numbers

Fault Tolerance and Availability

Understanding TIME in Distributed Systems

Data Centers for software engineers

Scaling Distributed Systems

Load Balancers

Structured Storage at Scale

Sharding in Storage Systems

SQL vs NO-SQL

Blob Storage at Scale

Caching in Distributed Systems

Queues and Asynchronous Processing

Data Analytics and Batch Processing

Logging and Monitoring

Searching at Scale

Conclusion

Routing Algorithms for Load Balancers

Routing in Load Balancers