Design of a Rate Limiter
Discover the detailed System Design required to build a highly available rate limiter. Learn how to structure components such as the decision-maker and the cache to manage traffic and mitigate race conditions. Implement online and offline updates to ensure low latency and system stability.
High-level design
A rate limiter is a dedicated service that sits between the client and the web server. It evaluates incoming requests against defined rules to decide whether to forward them to the server or block them. The figure below illustrates this interaction. Lyft’s rate-limiting service provides a practical example of how these rules are structured.
In this example, the unit is set to day and request_per_unit is 5. This rule limits the client to five marketing messages per day.
Detailed design
The high-level design leaves key questions unanswered:
Where are the rules stored? ...