Back To Module Home

Distributed Systems for Practitioners

0% completed

Introduction to Distributed Systems

Getting Started Fallacies of Distributed Computing Difficulties Designing Distributed Systems Measures of Correctness in Distributed Systems System Models Types of Failures The Tale of Exactly-Once Semantics Failure in the World of Distributed Systems Stateless and Stateful Systems Quiz

Basic Concepts and Theorems

Partitioning Algorithms for Horizontal Partitioning Replication Single-Master Replication Algorithm Multi-Master Replication Algorithm Quorums in Distributed Systems Safety Guarantees in Distributed Systems ACID Transactions The CAP Theorem Consistency Models CAP Theorem's Consistency Model Isolation Levels and Anomalies Prevention of Anomalies in Isolation Levels Consistency and Isolation Hierarchy of Models Why All the Formalities?Quiz

Distributed Transactions

Introduction to Distributed Transactions Quiz

Achieving Isolation

Achieving Serializability Pessimistic Concurrency Control (PCC)Optimistic Concurrency Control (OCC)Achieving Snapshot Isolation Achieving Full Serializable Snapshot Isolation Quiz

Achieving Atomicity

Hard to Guarantee Atomicity 2-Phase Commit (2PC)3-Phase Commit (3PC)Quorum-Based Commit Protocol Quiz

Concluding Distributed Transactions

How It All Fits Together Long-Lived Transactions and Sagas

Consensus

Defining the Consensus Problem FLP Impossibility The Paxos Algorithm Intricacies of Paxos Paxos in Real Life Replicated State Machine via Consensus Distributed Transactions via Consensus An introduction to Raft Communication among Raft Nodes Raft's Implementation Standing on the Shoulders of Giants Quiz

Time

What is Different in a Distributed System?A Practical Perspective A Theoretical Perspective Logical Clocks Quiz

Order

Total and Partial Ordering The Concept of Causality Lamport Clocks Vector Clocks Version Vectors Dotted Version Vectors Distributed Snapshot Problem Solving the Distributed Snapshot Problem Physical and Logical Time: Closing Thoughts Quiz

Networking

Introduction The Physical Layer The Link Layer - Services The Link-Layer Protocols The Network Layer The Transport Layer The Application Layer Taking a Step Back Quiz

Security

Introduction Authentication Confidentiality Integrity A Cryptography Primer Symmetric/Asymmetric Encryption and Digital Signatures Quiz

Security Protocols

Transport Layer Security (TLS)Public-Key Infrastructure (PKI)Web of Trust (PGP)OAuth Protocol Quiz

From Theory to Practice

Case Study 1: Distributed File Systems

Hadoop Distributed File System and Google File System Creating and Reading Files Writing and Deleting Files GFS Consistency Model Quiz

Case Study 2: Distributed Coordination Service

Coordination Service Zookeeper Guarantees Provided by Zookeeper Zookeeper's ZAB Protocol Examples of Powerful Primitives by Zookeeper’s API Quiz

Case Study 3: Distributed Data Stores

BigTable/HBase Architecture

Appends and Read Operations in HBase

Guarantees Provided by HBase

Cassandra's Data Model

Cassandra's Cluster Internode Communication

Cassandra's Consistency Levels

Linearizability Violations in Cassandra

Linearizability Guarantees by Cassandra

Cassandra Performing Queries Efficiently

Spanner's Data Model

Spanner's Architecture

Spanner using TrueTime

Spanner Operations

Case Study 4: Distributed Messaging System

Introduction to Kafka Kafka Levers Kafka's Messaging Guarantees Transactions, Storage Layout, and other Guarantees Quiz

Case Study 5: Distributed Cluster Management

Kubernetes Components of Master and Worker Nodes Quiz

Case Study 6: Distributed Ledger

Introduction to Corda Corda's Data Model Corda's Architecture Backwards Compatibility provided by Corda Quiz

Case Study 7: Distributed Data Processing Systems

Introduction Introduction to MapReduce MapReduce's Master-Worker Architecture Introduction to Apache Spark DAG of Stages in Apache Spark Perks of Apache Spark Apache Flink Time and Watermarks in Flink Failure Recovery in Flink Quiz

Practices & Patterns

Communication Patterns

Creating and Parsing Data Transfer of Data Datastores for Asynchronous Communication Communication Models

Coordination Patterns

Coordination Patterns

Data Synchronisation

Data Synchronisation Event Sourcing Change Data Capture (CDC)

Shared-nothing Architectures

Sharing Problems and their Solution Benefits and Drawbacks

Distributed Locking

Leases in Distributed Systems Preventing Safety Risks in Leases

Compatibility Patterns

Backwards Compatibility Maintaining Backwards Compatibility

Dealing with Failure

Failure Handling Techniques Applying Failure Handling Techniques Retries Containing Impact of Failure Backpressure Reacting to Backpressure

Distributed Tracing

Recording Program's Execution

Conclusion

...

/

Spanner's Data Model

Spanner's Data Model

Let's study the data model of Spanner.

Spanner is a distributed data store initially developed internally by J. C. Corbett et al.J. C. Corbett et al., “Spanner: Google’s Globally-Distributed Database,” in Proceedings of OSDI 2012, 2012. and ...