Learn to design real machine learning systems with the help of several open-ended machine learning problems.

Become a Machine Learning Engineer

Learn to design real machine learning systems with the help of several open-ended machine learning problems commonly asked during interviews at big tech companies. You will start by identifying the problem statement, understanding scale and latency requirements and defining metrics. Then you will come up with the architecture, select models, gather data and finally execute and evaluate the models offline and online.

Grokking the Machine Learning Interview

# Why do you need diverse Tweets?

Let's assume that you adopted the following modelling option for your Twitter feed ranking system. The Tweet selection component will select one-hundred thousand Tweets for user A's Twitter feed. The stage one ranker will choose the top five-hundred most engaging Tweets for user A. The stage two ranker will then focus on assigning engagement probabilities to these Tweets with a higher degree of accuracy. Finally, the Tweets will be sorted according to the engagement probability scores and will be ready for display on the user's Twitter feed.   

Consider a scenario where the sorted list of Tweets has five consecutive posts by the same author. No, your ranking model hasn't gone bonkers! It has rightfully placed these tweets at the top because:
- The logged-in user and the Tweet's author have frequently interacted with each other's Tweets
- The logged-in user and the Tweet's author have a lot in common like hashtags followed and common followees
- The author is very influential, and their Tweets generally gain a lot of traction

> 📝 This scenario remains the same for any modelling option.

## Diversity in Tweets' authors

However, no matter how good of a friend the author is or how interesting their Tweets might be, user A would eventually get bored of seeing Tweets from the same author repeatedly. Hence, you need to introduce diversity with regards to the Tweets' author.

## Diversity in tweets' content
Another scenario where we might need to introduce diversity is the Tweet's content. For instance, if your sorted list of Tweets has four consecutive tweets that have videos in them, the user might feel that their feed has too many videos. 


# Introducing the repetition penalty
To rid the Twitter feed from a monotonous and repetitive outlook, we will introduce a repetition penalty for repeated Tweet authors and media content in the Tweet.

One way to introduce a repetition penalty could be to add a negative weight to the Tweet's score upon repetition. For instance, in the following diagram, whenever you see the author being repeated, you add a negative weight of $-0.1$ to the Tweet's score.
 




Learn methods to reduce monotony on the user's Twitter feed.