Business Machine Learning/

...

/

Model Performance

Model Performance

Check the model performance with a complex predictor matrix using regularization.

We'll cover the following...

Moving forward, let's check how the penalties affect the performance of our models. Using the (X, y) dataset, we have not seen many benefits of regularization. We can try the second dataset (X_overfit, y_overfit) and see the effect of all three types of regularization and compare the results to learn how regularization helps us control overfitting. Let's start with linear regression without regularization.

Python 3.8
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
lr_model = LinearRegression()
lr_cv_mean_mse = -cross_val_score(estimator=lr_model, X=X_overfit, y=y_overfit,
                                  cv=5, scoring='neg_mean_squared_error').mean()
lr_cv_mean_r2 = cross_val_score(estimator=lr_model, X=X_overfit, y=y_overfit, 
                                cv=5, scoring='r2').mean()
print("These are results from linear regression (cv=5) without regularization:")
print("The Ridge CV mean MSE: ",lr_cv_mean_mse)
print("The Ridge CV mean R^2: ", lr_cv_mean_r2)

Now, let’s use ridge instead of linear regression.

Ridge regression

Let's use αR=10\alpha{_R} = 10 ...