Introduction to Grid Search Method

What is the grid search method?

Grid search is a straightforward method for hyperparameter optimization in ML. It is a well-known approach (after the random search method) commonly employed by ML practitioners due to its simplicity and convenience of implementation. We’ll learn its theory and how to apply it in a simple ML project using the open-source Python library called scikit-learn.

The grid search procedure begins with the specification of a set of possible values for each hyperparameter. For instance, if the ML model includes two hyperparameters, one for the learning rate and one for the number of estimators, the learning rate can be set to 0.1, 0.01, or 0.001, and the number of estimators can be set to 10, 20, or 50.

Once the value range for each hyperparameter has been set, the grid search method will train and evaluate an ML model for each hyperparameter value combination. This can be computationally expensive, especially if there are a large number of hyperparameters and alternative values for each. Nonetheless, it is a straightforward and widely used method to find the best hyperparameters.