Designing a Distributed Cache

In this lesson, we will learn to design a distributed cache. We will also understand the tradeoffs and design choices that can occur while we progress in our journey towards developing a solution.

Design Requirements

Let us start by understanding the requirements of our solution.

Functional

Following are the functional requirements:

• Insert data: The user of distributed cache system must be able to insert an entry to the cache.
• Retrieve data: The user should be able to retrieve data corresponding to a specific key.

Non-functional requirements

We will consider the following non-functional requirements:

• High performance: The primary reason for the cache is to enable fast retrieval of data. Therefore, both the Insert and Retrieve operations must be fast.

• Scalable: The cache system should scale horizontally with no bottlenecks on an increasing number of requests.

• Highly available: The unavailability of the cache will put an extra burden on the database servers, which can also go down at peak load intervals. We also require our system to survive occasional failures of components/network/power outages.

• Consistency: Data stored on the cache servers should be consistent, i.e., different cache clients retrieving the same data from different cache servers (primary/secondary) should be up to date.

• Affordability: Ideally, the caching system should be designed from commodity hardware instead of spending a fortune over a supporting component within the design of a system.

API Design

The API design for this problem is sufficiently easy since there are only two basic operations.

The API call to perform insertion should look like this:

insert(key, value)

Where the value is the data stored against a unique key. We assume that the key is a string and the value is an object. This function will return an acknowledgment or an error depicting the problem at the server end.

The API call to retrieve data from the cache should look like this:

retrieve(key)

The key is used for finding the location of the value stored in a cache server. This call will return an object to the caller.