Ridge Regression
Explore Ridge Regression to improve linear models by adding regularization to reduce noise sensitivity from correlated features. Understand how to implement ridge regression with scikit-learn, use cross-validation to tune the alpha hyperparameter, and apply these concepts to build more robust predictive models.
We'll cover the following...
Chapter Goals:
- Learn about regularization in linear regression
- Learn about hyperparameter tuning using cross-validation
- Implement a cross-validated ridge regression model in scikit-learn
While ordinary least squares regression is a good way to fit a linear model onto a dataset, it relies on the fact that the dataset's features are each independent, i.e. uncorrelated. When many of the dataset features are linearly correlated, e.g. if a dataset has multiple features depicting the same price in different currencies, it makes the least squares regression model highly sensitive to noise in the data.
Because real life data tends to have noise, and will often have some linearly correlated features in the dataset, we combat this by performing regularization. For ordinary least squares regression, the goal is to find the weights (coefficients) for the linear model that minimize the sum of squared residuals:
...