How Does Sequential Model-Based Optimisation Work

The following list is the optimization steps and procedures that the SMBO method must follow to find the combination of hyperparameter values that produce the best ML model performance.

The steps for SMBO are as follows:

1. Define the optimization problem.

2. Define the hyperparameter search space.

3. Choose the acquisition function.

4. Train the probabilistic model.

5. Select the next combination to evaluate.

6. Evaluate the objective function.

7. Update the probabilistic model.

8. Terminate the process.

1. Define the optimization problem

The first step is to define the optimization problem, which means stating the objective function to be optimized and the hyperparameters to be fine-tuned. In this step, the objective function is usually a performance evaluation metric such as accuracy for classification problems or mean absolute error for regression problems. The aim of the objective function is either to maximize or minimize the final score. The hyperparameters are the variables that can be changed to optimize this evaluation metric.