Back To Course Home

Copy-System Design-The Missing Pieces

0% completed

Introduction

System Design - The Missing Pieces Course Audience

Abstractions

Why are Abstractions Important?Network Abstractions: Remote Procedure Calls Spectrum of Consistency Models Spectrum of Failure Models

Non-functional System Characteristics

Availability Reliability Scalability Maintainability Fault Tolerance

Back-of-the-Envelope Calculations

Put Back-of-the-Envelope Numbers in Perspective Examples of Resource Estimation

Building Blocks

Domain Name System (DNS)

DNS new lesson Domain Name System

Sequencer

Introduction Design of Unique ID Generator Unique IDs with Causality

Rate Limiter

System Design: Rate Limiter Requirements of a Rate Limiter Design of a Rate Limiter Rate Limiter Algorithms Quiz on Rate Limiter

Distributed Cache

Introduction to Distributed Cache USE THIS LINK Introduction to Distributed Cache Clone Designing a Distributed Cache Memcached vs. Redis

Blob Store

Introduction to Blob Store clone Introduction to Blob Store Design of Blob Store Design Considerations in a Blob Store Evaluation of Blob Store Design Quiz on Blob Store

Content Delivery Network (CDN)

System Design: CDN Introduction to CDN Design of CDN In-depth Investigation on CDN (Part 1)In-depth Investigation on CDN (Part 2)Quiz on CDN

Load Balancers

Introduction to Load Balancers USE THIS LINK Introduction to Load Balancers Clone Advanced Details of Load Balancers

Key-Value Store

Introduction to Key-Value Store [clone]Introduction to Key-Value Store Ensuring Scalability and Replication Versioning Data and Achieving Configurability Enabling Fault Tolerance and Failure Detection

Distributed Messaging Queue

System Design: Distributed Messaging Queue Requirements of Distributed Messaging Queue Design Considerations in Distributed Messaging Queue Design of Distributed Messaging Queue (Part 1)Design of Distributed Messaging Queue (Part 2)Evaluation of Distributed Messaging Queue Design Quiz on Distributed Messaging Queue

Pub-sub

Introduction to Pub-sub [clone]Introduction to Pub-sub Designing the Pub-sub System

Distributed Task Scheduler

Introduction to Distributed Task Scheduler [clone]Introduction to Distributed Task Scheduler Design of Distributed Task Scheduler Design Considerations in a Distributed Task Scheduler Evaluation of Distributed Task Scheduler Design

Distributed Search

Introduction to Distributed Search [clone]Introduction to Distributed Search Indexing Design of Distributed Search Scaling Search and Indexing Evaluation of Distributed Search Design

Distributed Logging

Introduction to Distributed Logging [clone]Introduction to Distributed Logging Design of Distributed Logging Service

Distributed Monitoring

Introduction to Distributed Monitoring [clone]Introduction to Distributed Monitoring Pre-requisites of Monitoring System

Monitoring Server Side Errors

Designing a Monitoring System Detailed Design of Monitoring System Creative Way of Visualizing Data

Monitoring Client Side Errors

Focusing on Client Side Errors Designing a Monitoring System

Databases

Types of Databases [clone]Types of Databases Data Replication Partitioning Cost Benefit Analyses

Sharded Counters

System Design: Sharded Counters High-Level Design of Sharded Counters Detailed Design of Sharded Counters Quiz on Sharded Counters

Concluding Building Blocks

Conclusion RESTDEV Approach

Design YouTube

System Design: YouTube Requirements for YouTube Design Design of YouTube Evaluation of YouTube Design The Reality is More Complicated Quiz on YouTube

Design Quora

System Design: Quora Requirements for Quora Design Initial Design of Quora Final Design of Quora Evaluation of Quora Design

Design Google Maps

Introduction to Google Maps [clone]Introduction to Google Maps Design of Google Maps Challenges in Google Maps System Design Detailed Design of Google Maps Evaluation of Google Maps Design

Designing a Proximity Server like Yelp

Introduction to Yelp [clone]Introduction to Yelp Design of Yelp Design Considerations in Yelp Quiz on Yelp

Design Uber

System Design: Uber Requirements for Uber Design High-level Design of Uber Detailed Design of Uber Payment Service and Fraud Detection Evaluation of Uber Design Quiz on Uber

Design Twitter

System Design: Twitter Requirements for Twitter High-Level Design of Twitter Detailed Design of Twitter Client-side Load Balancer for Twitter Quiz on Twitter

Newsfeed System

System Design: Newsfeed System Requirements for Newsfeed System Design of Newsfeed System Evaluation of the Newsfeed System

Design Instagram

Introduction to Instagram [clone]Introduction to Instagram Design of Instagram Detailed Design of Instagram Quiz on Instagram

Design URL Shortening Service / TinyURL

System Design: TinyURL Requirements of TinyURL Design and Deployment of TinyURL Encoder for TinyURL Evaluation of TinyURL Quiz on TinyURL

Design a Web Crawler

System Design: Web crawler Requirements of Web Crawler Design of Web Crawler Design Improvements of Web Crawler Evaluation of Web Crawler

Design WhatsApp

Requirements of WhatsApp Design [clone]Requirements of WhatsApp Design WhatsApp High-level Design WhatsApp Detailed Design WhatsApp Design Evaluation Quiz on WhatsApp Design

Design Typeahead Suggestion

Introduction to Typeahead Suggestion System [clone]Introduction to Typeahead Suggestion System High-level Design of Typeahead Suggestion System Data Structure for Storing Prefixes Detailed Design of Typeahead Suggestion System Design Evaluation of the Typeahead Suggestion System Quiz on Typeahead Suggestion Design

Design Collaborative Document Editing Service / Google Docs

System Design: Google Docs Requirements of Google Docs Design Design of Google Docs Concurrency in Collaborative Editing Evaluation of Google Docs Design

Spectacular Failures

Introduction to Distributed System Failures Facebook, WhatsApp, Instagram, Oculus Outage AWS Kinesis Outage Affecting Many Organizations AWS Wide Spread Outage

Concluding Remarks

Appendix: System Design Interviews

What is a System Design Interview?How to Prepare for Success How to Perform Well

All content below this will likely go away

Design Exercises

Archived temporary lessons

Introduction to Distributed Logging [backup]Introduction to Distributed Monitoring [backup]Introduction to Pub-sub [backup]Requirements of TinyURL [backup]Introduction to Instagram [backup]Introduction to Web Crawler [backup]Introduction to Yelp [backup]Introduction to Key-Value Store [backup]Requirements of WhatsApp Design [backup]Introduction to Distributed Messaging Queue [backup]Introduction to Typeahead Suggestion System [backup]Copy How to Perform Well?Rate Limiter [backup]Introduction to CDN [backup]Types of Databases [backup]Introduction to Sharded Counters [backup]Introduction to Uber [backup]Introduction to Google Maps [backup]Introduction to Distributed Search [backup]Introduction to Distributed Task Scheduler [backup]Introduction to Blob Store [backup]AWS Kinesis Outage Affecting Many Organizations - 2020-11-25 COPY Copy How to Prepare for Success?Copy What is a System Design Interview?AWS Outage 2021-12-15 Y Z Resources for Further Exploration Facebook, WhatsApp, Instagram, Oculus Outage - 2021-10-04 COPY AWS Wide Spread Outage 2021-12-07 COPY Summary F EvaluationOld Coping with the Challenges A Requirements and Challenges Implementation: Elastic Search Drawbacks of Colocated Indexing and Search Client-side Load Balancers (clone)Enabling Fault Tolerance and Failure Detection clone Designing pub-sub Backup of Design and Deployment [Option] Further Concepts fourth-vertical sharding (To be removed)Introduction Copy-Replication sixth-automatic sharding (to be archived for later use)copy- Partitioning Messaging Queue-Second Draft Messaging Queue-References Messaging queue-Illustrations and other stuffs backup Messaging Queue-Initial Draft A New Perspective: Favoring Consistency WhatsApp References and backup

Design Resource Allocator for a Large Datacenter

A B C D E F G Lesson-01: Resource Management Lesson-02: Challenges in Resource Management Lesson-03: RAS Design Lesson-04: RAS Evaluation Facebook: Optimized Datacenter Resource Allowance System RAS: Continuously Optimized Region-Wide Datacenter Allocation

Design Zoom

Continuous Monitoring using Data Processing

Batch Processing Stream Processing Data Encoding

Design Live Commenting at Facebook

Security

Row Hammer Problem Modern Processors Doing Faulty Calculations Occasionally

For Noor: Placeholder for Illustration Making

D E F G H I J K L Illustrations for Ibrahim Illustrations for Fareeha Illustrations for Bismillah Illustrations for Asmat Illustrations for Ahsan Illustrations for Hamza Illustrations for Ateeq

Appendix

Related Educative Courses Ideas Yet to Expand

Backup of our Lessons

Backup of Evaluation Twitter Design clone Introduction to Newsfeed System [backup]Replicating the Data in Key-Value Store [merged with scalability]Achieving Configurablility [merged with versioning]Detecting Failures [merged with fault tolerance]Test Yourself [merged with Fault tolerance]clone 2 [sir's key-value store]Uber Design clone clone 3 [sir's key-value store]Prep for Quora Copy of Put Back-of-the-Envelope Numbers in Perspective Extras Copy of Examples of Resource Estimation Case Study: S3 The Reality is More Complicated: Azure Designing pub-sub Design Considerations [to be removed]Introduction-monitoring-clone Notes on collaborative document editing s3 clone monitor clone Logging at Pinterest [clone]Designing Distributed Logging Service [clone]Distributed Logging [clone]Copy of domain name system Copy of Load Balancers Content Delivery Network (CDN)Copy of Load Balancers (on 2022-01-11)Types of Databases Data Replication Models Horizontal Sharding of Database Vertical Sharding of Database Cost Benefit Analysis of Centralised and Distributed Database Automatic Sharding and Replication of Data at the Global Scale Distributed Unique ID Generator (Sequencer)Focusing on Causality X Sir's content about unique ID generator Copy2 of Content Delivery Network (CDN)Rate-limiter References Sharded counter copy Copy - Monitoring a Service's Availability Copy - Use-case of Monitoring a Service Copy - Controlling the Side Effects of Monitoring

Caching Billions of Tiny Objects on Flash

Draft Lesson Designing a Caching Layer for Tiny Objects B C D Tiny Objects and Flash Designing the System Flash Caches Improving Miss-ratio How RRIParoo Works Proving Kangaroo Works Efficiently

Design Quora

Use of Public Cloud

Webserver Architecture

Periodic Job Running

Designing Logging and Aggregation System

Economical Fault Tolerance

Scaling Database

Asynchronously Handling Billions of Tasks Per Day

Copy-Design YouTube

Copy-Requirements Copy-Design Copy-Evaluation Copy-The Reality is More Compllicated Copy-Quiz

Identity & Access Management

Identity and Access Management X Y Z

Copy of CDN (02-03-2022)

Copy introduction Copy Design Copy Quiz

Conclusion

As we saw in this design activity, a modern distributed service touches on most distributed system concepts such as scaling, availability, fault tolerance, data processing, etc.

We can manage the complexity by creating independent sub-services and delegating work to them.

Create a free account to access the full course.

By signing up, you agree to Educative's Terms of Service and Privacy Policy