Feature Selection: Intrinsic Methods
Explore intrinsic feature selection methods that embed feature importance within model training. Understand regularization techniques such as Ridge, Lasso, and ElasticNet, and decision tree-based methods for selecting influential features. Learn to apply scikit-learn's SelectFromModel for efficient feature reduction.
We'll cover the following...
Intrinsic or embedded methods
Embedded methods learn about the features that contribute the most to the model’s performance while the model is being created. You have seen feature selection methods in the previous lessons, and we will discuss several more in future lessons, like decision tree-based methods.
-
Ridge regression (L2-regularization)
-
Lasso regression (L1-regularization)
-
ElasticNet regression (uses both L1 and L2 regularization)
-
Decision tree-based methods (decision tree classification, random forest Classification, XgBoost classification, LightGBM).
We know regularization reduces some of the parameters in the equation below to zero. This property of regularization methods can be used as a feature selection method.
...