This course immerses the learner into the design of systems and makes them ready for any new design problems.

sys-design.tar.gz

basic

This course provides an alternate perspective to system design where we learn about building blocs first and then use them to build bigger, more capable systems.

Copy-System Design-The Missing Pieces

# High-level design

The high-level design shows how we will interconnect various components we identified in the previous lesson. We have started developing a solution to support the functional and non-functional requirements with this design.

The workflow for the abstract design is provided below:

1. The user uploads a video to the server.
2. The server stores the metadata and the accompanying user data to the database and, at the same time, hands over the video to the encoder for encoding (see 2.1 and 2.2 in the illustration above).
3. The encoder along with the transcoder, compresses the video and transforms it into multiple qualities (like 2160p, 1440p, 1080p, etc.). The videos are stored on blob storage (similar to GFS or S3).
4. Some popular videos may be forwarded to Content Distribution Network (CDN) which acts as a cache.
5. The CDN, because of its vicinity to the user, lets the user stream the video with low latency. However, CDN is not the only infrastructure for serving videos to the end-user as we will see in the detailed design.

There are several reasons why it is a good idea to introduce a server in between the encoder and the client:
- The client can be malicious and can abuse the encoder.
- If the uploaded video is a duplicate server could filter it out.
- Encoders will be available on a private IP address within YouTube's network and not available for public access.


There are several reasons why it is a good idea to introduce a server in between the encoder and the client:
<ul>
<li>The client can be malicious and can abuse the encoder.</li>
<li>If the uploaded video is a duplicate server could filter it out.</li>
<li>Encoders will be available on a private IP address within YouTube’s network and not available for public access.</li>
</ul>

Why don't we upload the video directly to the encoder (instead of the server)? Isn't the current strategy introducing an additional delay?

Why don’t we upload the video directly to the encoder (instead of the server)? Isn’t the current strategy introducing an additional delay?

Quiz

Quiz

# API design

Let's understand the design of APIs in terms of the functionalities we are providing. We will design APIs to translate our feature set into technical specifications. In this case, the REST APIs can be used for simplicity and speed purposes. Our API design section will help us understand how the client will request services from the backend application of YouTube. Let's develop APIs for each of the following features:
- Upload video
- Stream video
- Search video
- View thumbnails
- Like/dislike video
- Comment video


## Upload video

The POST method can upload a video to the  `/uploadVideo` API:

```
uploadVideo(user_id, video_file, category_id, title, description, tags, default_language, privacy_settings)
```
We would like to explain the following parameters here:

- `user_id` is the user that is uploading the video.
- `video_file` is the video file that the user wants to upload.

>The video file is broken down into smaller packets and uploaded to the server in order. In case of failure, YouTube can store data for a limited time and resume the upload if the user retries. To understand the concept in detail, you may want to read about #key# Asynchronous APIs: The kind of APIs where the server cannot fulfill the request immediately either because the data is too large or isn't readily available. In that case, the client gets an immediate response that the server has accepted the request while the server responds with the data later on. Such a mechanism saves CPU and network bandwidth and does not keep the client waiting. #key#.
- `category_id` refers to the category a video belongs to. Typical categories can be _Entertainment_, _Engineering_, _Science_, etc.
- `title` is the title of the video.
- `description` is the description of a video.
- `tags` refer to the specific topics the content of the video covers. Tags can improve search results.
- `default_language` is the default language a page will show to the user when the video is streamed.
- `privacy_settings` refers to the privacy of the video. Generally, videos can be a public asset or private to the uploader.

## Stream video
The GET method is best suited for the `/streamVideo` API in the following way:

```
streamVideo(user_id, video_id, screen_resolution, user_bitrate, device_chipset)
```
Some new things introduced in this case are:
- `screen_resolution` the server can best optimize the video if the user's screen resolution is known.
- `user_bitrate` the transmission capacity of the user is required to understand which quality of video chunks should be transferred to the client/user.
>The server will store different qualities of the same video in its storage and serve users based on their transmission rate.
- `device_chipset` many YouTube users watch content over handheld devices which makes it important to know the handling capabilities of these devices to better serve the users.

## Search videos
The `/searchVideo` API will use GET method and have the following usage:

```
searchVideo(user_id, search_string, length, quality, upload_date)
```
- `search_string` is the string used for searching videos by their title.

Other attributes can be optional:
- `length` to filter videos based on their length in terms of time.
- `quality` to filter videos based on the resolution like 2048p, 1440p, 1080p, etc.
- `upload_date` to filter videos based on their date of uploading to the YouTube platform.

## View thumbnails
We can use GET method to access the `/viewThumbnails` API:

```-
viewThumbnails(user_id, video_id)
```
This API will return the thumbnails of a video in a sequence.

## Like/dislike video
The like/dislike API will use the GET method. As shown below, it is fairly simple:
```
likeDislike(user_id, video_id, like)
```
We can use the same API for like/dislike functionality. Depending on what is passed as a parameter to the `like` field, we can update the database accordingly. i.e., `0` for like and a `1` for dislike.

## Comment video
Much like the like/dislike API, we only have to provide the comment string to the API. This API will also use the GET method.

```
commentVideo(user_id, video_id, comment_text)
```

# Database schema

Each of the above features in the API design requires support from the database. i.e., we will need to store the above details in our database schema to provide services to the API gateway.



Take a deep dive into the design of YouTube.

Design of YouTube

High-level design

API design

Upload video