Hyperparameters and Their Tuning
Explore the concept of hyperparameters and their role in decision tree learning. Learn to use scikit-learn's GridSearchCV and RandomizedSearchCV methods to find optimal hyperparameter values, enhance model accuracy, and avoid overfitting. Understand evaluation techniques and how to save the best parameters for effective predictions.
We'll cover the following...
Let’s talk about hyperparameters and their tuning. A parameter is called a hyperparameter if its value is set before we start the training process of our model. Tuning is simply finding the best set of these parameters. Hyperparameters are not directly learned within the models. However, the values of other parameters are derived via training the model.
Scikit-learn provides two commonly used approaches to search for the values of hyperparameters:
GridSearchCV: Considers all the combinations for the provided values.
RandomisedSearchCV: Can sample a given number of candidate values from a parameter space with a specified distribution.
It’s important to note that both the randomized search and the grid search explore the same parameter space. However, randomized search has a drastically lower run ...