sixth-automatic sharding (to be archived for later use)

What is Spanner?

Spanner is Google’s scalable, multi-version, globally distributed, and synchronously-replicated database. It is the first system to distribute data at a global scale and support externally consistently distributed transactions. Being externally consistent basically says that the commit order of transactions is the same as the order in which you actually appear in which users actually see the transactions executed with respect to global wall clock time and spanner is the first system to provide this property at a global scale.

The goal is to achieve very strong consistency properties in the global database. The focus is on serializable transaction isolation, linearizability for reads and writes, and sharding.

When we have a huge amount of data that cannot be stored on a single node, we split it into subsets of data. This is called sharding. Now the data is distributed across all different nodes. Sharding raises a concern that the data should be updated on all data nodes for any read/write transaction. It is a challenging task to do in a distributed system. In such a system, we need atomic commits like be committed on all nodes or none.

How Spanner achieves strong consistency?

Spanner implements these properties using the Paxos consensus algorithm to ensure atomic commits, two-phase locking for serializability, and two-phase commit for cross-shard atomicity. Two-phase locking means for any read we take a shared lock on the data we want to read and for any writes, we’d need to take an exclusive lock on the data we want to write and hold those locks until the transaction commits. Spanner also utilizes a global clock called true-time. It exposes uncertainty in the clock by representing time as an interval.

What new thing does Spanner provide?

The exciting thing about Spanner is that it allows read-only transactions without needing any locks. It is essential because, in two-phase locking, we need a shared lock to read any data. So if we need to read a large amount of data at once, we will be taking a lot of time, and other transactions to queries have to wait, which is not ideal.