Unbalanced Capacities
Explore the challenges of unbalanced capacities in distributed systems where one service can overwhelm another. Understand how to predict demand mismatches and apply strategies like circuit breakers, handshaking, backpressure, and bulkheads to build resilient, fault-tolerant systems.
We'll cover the following...
Mismatched ratios
Whether our resources take months, weeks, or seconds to provision, we can end up with mismatched ratios between different layers. That makes it possible for one tier or service to flood another with requests beyond its capacity. This especially holds when we deal with calls to rate-limited or throttled APIs.
In the illustration below, the front-end service has 3,000 request-handling threads available. During peak usage, the majority of these will be serving product catalog pages or search results. Some ...