Understand model techniques along with the best practices in applying ML models in production at scale.

Deep Dive into System Design Interview

Machine learning system design is an important component of any machine learning interview. The ability to address problems, identify requirements, and discuss trade-offs can help us stand out among hundreds of other candidates. This module will discuss model techniques, along with best practices in applying scalable machine learning models in production. Once we're done with this module, we’ll be able to apply and leverage knowledge from top researchers at tech companies. We’ll have an up-to-date knowledge of model techniques from hundreds of the latest research and industry papers. We might even be able to impress interviewers once we’ve gone through the lessons of this module.

Machine Learning System Design

# Video recommendations
## 1. Problem statement
Build a video recommendation system that personalizes content for YouTube users. The main goal is to maximize user engagement by recommending videos they are likely to watch and enjoy. But it doesn't stop there—we also want to introduce new and diverse content, not just more of what users already watch.
> Think of it like this: If someone watches cooking videos all day, the system shouldn’t just give them more of the same. It should also suggest travel vlogs, tech reviews, or short films they might find interesting.

**Goals:**
* Personalized recommendations based on user behavior

* Increase user engagement (watch time, click-throughs, conversions)

* Promote fresh content and not just historical favorites







## 2. Metrics design and requirements

### Metrics
Choosing the right metrics is crucial. You can’t improve what you don’t measure. We split our metrics into offline and online evaluation methods:
#### Offline metrics 
Used during model training and validation.

* **#key# Precision: The fraction of relevant instances among the retrieved instances. #key# & #key# recall: The fraction of the total amount of relevant instances that were actually retrieved. #key#:** How accurate and complete are our recommendations?

* **Ranking loss:** Measures how well the system ranks relevant videos higher.

* **Log loss:** Captures the model's confidence in its predictions.

> Offline metrics are useful for rapid experimentation, but they don’t always reflect real user behavior.

#### Online metrics
These measure actual user behavior on the platform.

* **Click-through rate (CTR):** Do users click the videos we recommend?

* **Watch time:** How long do they stay engaged with the content?

* **Conversion rate:** Do they take desired actions like subscribing, liking, or sharing?

> A/B tests help validate whether your model performs well in the real world—not just in training.  


### Requirements
A robust video recommendation system must operate under demanding conditions. Here's what’s expected:
### Training 
* **Challenge:** User interests shift rapidly. Videos can go viral in hours.

* **Solution:** The system should support frequent retraining (multiple times per day) to capture these trends.

### Inference 
- **Challenge:** When a user opens the homepage, the system must instantly recommend ~100 videos.

* **Solution:** Response time (latency) should be under 200ms, ideally under 100ms.

> Fast recommendations are essential to keep users engaged. No one wants to wait for suggestions to load.

### Exploration vs. Exploitation: Finding the Right Balance
A common challenge in recommendation systems is balancing:

* **Exploitation:** Recommending videos similar to what users already like

* **Exploration:** Suggesting new, unfamiliar content that might surprise or delight them

Too much exploitation = repetitive, boring feed
Too much exploration = irrelevant recommendations

> Great recommendation systems strike a balance—relevance plus discovery.

## Summary

|**Type**|**Desired goals**|
|:-: |:-: |
|Metrics|Reasonable precision, high recall|
|Training|High throughput with the ability to retrain many times per day|
|Inference|Latency from 100ms to 200ms|
||Flexible to control exploration versus exploitation|

Learn about the problem statement and metrics for building a video recommendation system.

Problem Statement and Metrics

Video recommendations

1. Problem