System Design: The Key-value Store
Let's understand the basics of designing a key-value store.
We'll cover the following
Introduction to key-value stores
Key-value stores are
Usually, it’s preferred to keep the size of value relatively smaller (KB to MB). We can put large data in the blob store and put links to that data in the value field. Key-value stores are useful in many situations, such as storing user sessions in a web application and building NoSQL databases.
It’s challenging to scale traditional databases with strong consistency and high availability in a distributed environment. Many real-world services like Amazon, Facebook, Instagram, Netflix, and many more use primary-key access to a data store instead of
Note: Many applications might not require a rich programming model provided by a traditional relational database management system (RDBMS). Using RDBMS for such applications is often expensive in terms of cost and performance.
How will we design a key-value store?
We’ve divided the key-value system design into the following four lessons:
- Design a Key-value Store: We’ll define the requirements of a key-value store and design the API.
- Ensure Scalability and Replication: We’ll learn to achieve scalability using consistent hashing and replicate the partitioned data.
- Versioning Data and Achieving Configurability: We’ll learn to resolve conflicts that occur due to changes made by more than one entity, and we’ll make our system more configurable for different use cases.
- Enable Fault Tolerance and Failure Detection: We’ll learn to make a key-value store fault tolerant and how to detect failures in the system.