Overfitting, Underfitting, and Generalization
Explore how overfitting and underfitting influence machine learning models and discover methods to ensure generalization on new data. This lesson helps you understand model complexity, training versus test errors, and practical steps to balance flexibility for reliable predictions.
We'll cover the following...
A delivery-time model can predict historical routes almost perfectly and still make poor predictions for new routes. For example, a curve might bend sharply to pass through every recorded distance and delivery time, including delays caused by roadworks or an unusually slow pickup. Those accidents occurred in the past data, but they do not reliably repeat.
Overfitting happens when a model learns details of its training data that are noise rather than durable patterns. Training error can become extremely low because the model memorizes those details. Underfitting happens when a model is too simple to capture the pattern that matters. A straight line may miss a real curved relationship between route distance and delivery time.
The goal is generalization, meaning accurate performance on unseen data from the same kind of problem. A training score tells us how closely a model fits records it has already seen. It does not answer the delivery question that matters, which is how close the model will be on a new route.
Curve flexibility and noise
A polynomial regression model can produce curves with different amounts of flexibility. A degree 1 polynomial produces a straight line. A degree 3 polynomial can bend gradually. A degree 10 polynomial can bend repeatedly to follow individual points.
The delivery-time-like data below contains a ...