L1 (Lasso) and L2 (Ridge) Regularization

Marketing features such as search_ads_spend, social_ads_spend, and email_campaigns often move together. When two predictors carry nearly the same information, an ordinary linear regression model can assign a large positive coefficient to one and a large negative coefficient to the other. A small change in the training sample can then produce a very different set of coefficients, even when predictions look similar.

Regularization adds a penalty for large coefficients while the model fits the training data. The penalty makes the model less free to rely on extreme weights, which can reduce overfitting and improve performance on unseen records. It changes the fitting objective, not the input data or the meaning of the target.

Ridge and Lasso apply different penalties. Ridge uses L2 regularization, the sum of squared coefficients.

Lasso uses L1 regularization, the sum of absolute coefficient values.

In both formulas, $\alpha$ controls penalty strength. When $\alpha$ ...