Metrics
Explore how to evaluate ad prediction systems using offline and online metrics. Learn to compare models with log loss and assess real-world impact through revenue and user engagement tracking. Understand the importance of balancing advertiser value and user experience for sustainable platform success.
We'll cover the following...
The metrics used in our ad prediction system will help select the best machine-learned models to show relevant ads to the user. They should also ensure that these models help the overall improvement of the platform, increase revenue, and provide value for the advertisers.
Like any other optimization problem, there are two types of metrics to measure the effectiveness of our ad prediction system:
- Offline metrics
- Online metrics
📝 Why are both online and offline metrics important?
Offline metrics are mainly used to compare the models offline quickly and see which one gives the best result. Online metrics are used to validate the model for an end-to-end system to see how the revenue and engagement rate improve before making the final decision to launch the model.
Offline metrics
As ...