Metrics
Explore how to select and use both offline and online metrics for ad prediction systems. Understand the importance of log loss for model calibration and how to track revenue, engagement, and counter metrics to ensure system effectiveness and user satisfaction.
We'll cover the following...
The metrics used in our ad prediction system will help select the best machine-learned models to show relevant ads to the user. They should also ensure that these models help the overall improvement of the platform, increase revenue, and provide value for the advertisers.
Like any other optimization problem, there are two types of metrics to measure the effectiveness of our ad prediction system:
- Offline metrics
- Online metrics
📝 Why are both online and offline metrics important?
Offline metrics are mainly used to compare the models offline quickly and see which one gives the best result. Online metrics are used to validate the model for an end-to-end system to see how the revenue and engagement rate improve before making the final decision to launch the model.
Offline metrics
As ...