Design Distributed Cache

Problem statement

A typical system consists of the following components:

• It has a client that requests the service.
• It has one or more service hosts that entertain client requests.
• It has a database used by the service for data storage.

Under normal circumstances, this abstraction performs fine. However, as the number of users increases, the database queries also increase. And as a result, the service providers are overburdened, resulting in slow performance.

In such cases, a cache is added to the system to deal with performance deterioration. A cache is a temporary data storage that can serve data faster by keeping data entries in memory. Caches store only the most frequently accessed data. When a request reaches the serving host, it retrieves data from the cache (cache hitWhen the requested data is found in the cache, the server responds with the data immediately.) and serves the user. However, if the data is unavailable in the cache (cache missWhen the requested data isn’t found in the cache, it’s called a cache miss.), the data will be queried from the database. Also, the cache is populated with the new value to avoid cache misses for the next time.

A cache is a nonpersistent storage area used to keep repeatedly read and written data, which provides the end user with lower latency. Therefore, a cache must serve data from a storage component that is fast, has enough storage, and is affordable in terms of dollar cost as we scale the caching service. The following illustration highlights the suitability of RAM as the raw building block for caching:

We understand the need for a cache and suitable storage hardware, but what is distributed cache? Let’s discuss this next.

What is a distributed cache?

A distributed cache is a caching system where multiple cache servers coordinate to store frequently accessed data. Distributed caches are needed in environments where a single cache server isn’t enough to store all the data. At the same time, it’s scalable and guarantees a higher degree of availability.

Caches are generally small, frequently accessed, short-term storage with fast read time. Caches use the locality of referenceLocality of reference is the notion that a program will access a particular set of data over a short period of time. Locality can be spatial or temporal. principle.

Generally, distributed caches are beneficial in the following ways:

• They minimize user-perceived latency by precalculating results and storing frequently accessed data.
• They pre-generate expensive queries from the database.
• They store user session data temporarily.
• They serve data from temporary storage even if the data store is down temporarily.
• Finally, they reduce network costs by serving data from local resources.

In this section, we’ll learn to design a distributed cache. We’ll also discuss the trade-offs and design choices that can occur while we progress in our journey towards developing a solution.

Requirements

Let us start by understanding the requirements of our solution.

Functional requirements

The following are the functional requirements: ...