Search⌘ K
AI Features
Home
Courses
Grokking Modern System Design Interview

System Design: The Key-Value Store

Define the core concept of a key-value store, explaining its function as a distributed hash table (DHT). Understand why these systems are essential for modern web-scale applications, offering high availability and scalability. Learn the foundational requirements for designing a reliable key-value system.

We'll cover the following...

Introduction to key-value stores

Key-value stores are distributed hash tables (DHTs)A distributed hash table (DHT) is a decentralized storage system that provides lookup and storage schemes similar to a hash table, storing key-value pairs. Source: https://www.educative.io/edpresso/what-is-a-distributed-hash-table. A unique key, generated by a hash function, binds to a specific value, and the store remains agnostic to the value’s structure. Values can be blobs, images, server names, or any user-defined data.

Values should generally be small (KB to MB). For large datasets, we can store the content in a blob store and save a link to it in the value field. Key-value stores are useful for scenarios like storing user sessions and building NoSQL databases.

Scaling traditional databases with strong consistency and high availability is challenging in distributed environments. Many services, such as Amazon, Facebook, and Netflix, use primary-key access instead of traditional online transaction processing (OLTP) databases.Online transaction processing (OLTP) involves gathering input information, processing the data, and updating existing data to reflect the collected and processed information. The OLTP databases carry out day-to-day tasks like insertion, updating, and deletion of the data in the database. The common use cases include bestseller lists, shopping carts, customer preferences, session management, sales rank, and product catalogs.

Note: Many applications do not require the rich programming model of a traditional relational database management system (RDBMS). Using an RDBMS for these scenarios is often expensive in terms of cost and performance.

How will we design a key-value store?

We divide the key-value system design into four lessons:

  1. Design a key-value store: Define requirements and design the API.

  2. Ensure scalability and replication: Achieve scalability using consistent hashing and replicate partitioned data.

  3. Versioning data and achieving configurability: Resolve conflicts caused by concurrent updates and configure the system for different use cases.

  4. Enable fault tolerance and failure detection: Implement fault tolerance and detect system failures.