Optimization Methods: Random Search

What is random search?

Random search is a simple and straightforward optimization algorithm that involves randomly generating potential solutions to a problem and evaluating their performance. It is a widely used optimization method in machine learning, engineering, and other fields where optimization problems arise.

In random search, the optimization process begins by defining a search space, which is the set of all possible solutions to the problem. The search space can be continuous, discrete, or a combination of both. Random search then randomly generates a set of candidate solutions from the search space. The number of candidate solutions and the range of values for each solution can be set by the user.

Each candidate solution is then evaluated using an objective function that measures how well the solution performs on the problem at hand. The objective function can be a simple performance metric or a more complex measure that takes multiple factors into account.

After evaluating all the candidate solutions, random search selects the solution with the best objective function value as the current best solution. This best solution is then used as the starting point for the next iteration of random search.

The optimization process continues for a predetermined number of iterations or until a stopping criterion is met. The stopping criterion can be based on the number of iterations, the improvement in the objective function value, or other factors.

Explaining random search

Let’s look at an example. In the following image, we are at point A and want to move ahead to point B.