Design Twitter

Learn about the system design for Twitter.

Overview

Twitter is a free microblogging social network with 397 million users as of 2021. One of the main reasons for Twitter’s popularity is the vast sharing of breaking news on the platform.

In this lesson, we will describe the functional and non-functional requirements of Twitter. Then we’ll look deeper into its API design and identify the different components of the Twitter architecture. In addition, we will discuss how to manage the Top K problem, such as Tweets liked or viewed by millions of users on Twitter. Finally, we’ll also explain how Twitter performs load balancing for its microservices system to manage billions of requests between the various service instances.

Requirements

Let’s go over the functional and non-functional requirements below.

Functional requirements

  • Registered users can post and delete one or more Tweets on Twitter.

  • They can like, dislike, and reply to Tweets.

  • Users can search tweets by using keywords, hashtags, or usernames.

  • Users can follow or unfollow other users.

  • Users can view other users’ timelines with their Tweets or their own home timelines with the Tweets of users they follow.

Non-functional requirements

  • Availability: Twitter’s service needs to be highly available.

  • Latency: The latency of the distribution of Tweets to followers must be low.

  • Scalability: Twitter’s workload is read-heavy with around a 1:1000 read-to-write ratio. So high storage capacity is needed to store and deliver Tweets posted by public figures to their millions of followers.

  • Reliability: Twitter needs to be highly reliable, and no uploaded content should get deleted or damaged.

  • Consistency: An effective technique is needed to offer rapid feedback to the user (who liked someone’s post), then to other specified users in the same region, and finally to all worldwide users linked to the Tweet.

Building blocks we will use

We will use the following building blocks in the Twitter design.

Building blocks used in Twitter design
  • The DNS is the service that maps human-friendly Twitter domain names to machine-readable IP addresses.
  • Load balancers distribute the read/write requests among the respective services.
  • Sequencers generate the unique IDs for the Tweets.
  • Databases store the metadata of Tweets and users.
  • Blob stores store the images and video clips attached with the Tweets.
  • Key-value stores are used for multiple purposes, such as indexing, identifying the specified counter to update its value, identifying the Tweets of a particular user, and so on.
  • Pub-sub is used for real-time processing, such as the elimination of redundant data, organizing data, and so on.
  • Sharded counters help to handle the count of multiple features, such as viewing, liking, Retweeting, and so on, of the accounts with millions of followers.
  • A cache is used to store the most requested and recent data in RAM to give users a quick response.
  • A CDN helps end users access the data with low latency.
  • Monitoring analyzes all outgoing and incoming traffic, identifies the redundancy in the storage system, figures out the failed node, and so on.

High-level design

Let’s begin with the high-level design of our Twitter system.

  • Users post Tweets delivered to the server through the load balancer. Then, the system stores it in persistent storage.

  • The DNS provides the specified IP address to the end user to start communication with the requested service.

  • The CDN is situated near the users to provide requested data with low latency. When users search for a specified term or tag, the system first searches in the CDN proxy servers containing the most frequently requested content. ...

Level up your interview prep. Join Educative to access 70+ hands-on prep courses.