In this lesson, you will look at different metrics that you can use to gauge the performance of the movie/show recommendation system.
Types of metrics
Like any other optimization problem, there are two types of metrics to measure the success of a movie/show recommendation system:
Online metrics are used to see the system’s performance through online evaluations on live data during an A/B test.
Offline metrics are used in offline evaluations, which simulate the model’s performance in the production environment.
We might train multiple models and tune and test them offline with the held-out test data (historical interaction of users with recommended media). If its performance gain is worth the engineering effort to bring it into a production environment, the best performing model will then be selected for an online A/B test on live data.