Comparing Bayesian Optimization with Other Optimization Methods

Bayesian optimization is a popular and effective method for global optimization of expensive black box functions. The following presents a brief comparison of Bayesian optimization with other methods.

Bayesian optimization vs. gradient descent

• Differentiability: Gradient descent methods require the objective function to be differentiable, while Bayesian optimization doesn’t have this requirement, which makes it suitable for optimizing a wider range of functions.

• Evaluation cost: Bayesian optimization is ideal for expensive black box functions because it seeks to minimize the number of function evaluations. Gradient descent ...