Ensuring High Availability in DynamoDB

Let's discuss some ways in which we can keep our database highly available.

Write availability

Our database follows a model where tables are partitioned, and partitions are replicated. A set of replicas of the same partition is called a replication group. Partitions use a leader election based on Multi-Paxos to elect a leader replica. This leader replica is responsible for serving to write requests—it adds the new entry to the write-ahead log and indexes in its memory. Later, this leader shares its write-ahead log and tree index with other partitions in the replication group. This is possible if enough healthy replicas (two-thirds) are available to form a write quorum. Write consistency in our distributed database depends on the ability of the network to maintain healthy leader partition replicas and enough healthy replicas in the replication group to form a write quorum. A minimum quorum is also required to elect a leader.

One way to maintain write availability is for the leader replica to add another member to the replication group as a log replicaA log replica only stores the write-ahead log. Log replicas are like acceptors in Paxos. as soon as it finds that some replica is unresponsive or faulty. This is the fastest way to maintain the minimum number of replicas required to form a quorum.