Shed Load
Explore load shedding as a critical stability pattern to maintain system responsiveness under uncontrollable demand. Understand how services can reject requests to avoid slowdowns and failures caused by resource exhaustion, improving system uptime and managing high load efficiently.
Increasing load
Services, microservices, websites, and open APIs all share one characteristic: they have zero control over their demand. At any moment, more than a billion devices could make a request. No matter how strong our load balancers or how fast we can scale, the world can always make more load than we can handle.
What happens at the network level
At the network level, TCP copes with a flood of connection attempts via the listen queue. Every incomplete connection goes into a queue per port. It’s up to the application to accept the connections. When the queue is full, new connection ...