System Design: The Distributed Cache
Explore the foundational role of caching in modern System Design to improve performance and reduce database load. Define a distributed cache and explain why distribution is essential for scalability and high availability. Identify common use cases and industry-standard solutions like Redis and Memcached.
Problem statement
A typical system consists of three core components:
The client who requests the service.
The service host processes client requests.
The database that stores the service’s data.
While this abstraction works for low traffic, scaling up the number of users increases database query volume. This overloads the database and causes high latency. To resolve this, we add a cache to the system.
A cache is a high-speed storage layer that temporarily holds data in memory to serve requests faster.
Caches store only the most frequently accessed data. When a request reaches the serving host, it retrieves data from the cache (
After a miss, the cache is populated with the new value to prevent future misses.
A cache stores transient, frequently accessed data to reduce latency for the end user. Therefore, the storage hardware must be fast, large enough to hold the working set, and cost-effective. RAM is the standard building block for caching due to its speed and efficiency.
The following illustration highlights the suitability of RAM for caching:
We understand ...