Getting Started with Optimization

To talk about optimization, we need to know what a function is. Functions are the focus of optimization.

Functions and optimization

A function is a correspondence that transforms an input into an output in a deterministic way. When we say deterministic, we mean that the same input will always be transformed into the same output.

For example, consider a function $f(x) = x^2$. Here, $f$ is the name of the function and the input will always be inside the parentheses. In this case, the input is denoted by $x$. The output is written after the equals sign. This is a function that transforms the input into its square. So, if the input is 1, then the output will be 1, if the input is 4, the output will be 16, if the input is 100, the output will be 10,000, and so on. We denote it as:

$$f(1) = 1$$

$$f(4) = 16$$

$$f(100) = 10000$$

Optimization is the field that studies how to find the minimum and/or maximum of a function. The minimum of a function is the input that corresponds to the smallest output. The maximum of a function—yes, you guessed right!—is the input that corresponds to the largest output. For our example function, the minimum would be zero, since $f(0) = 0$, and all other inputs will produce a bigger output. But, what’s the maximum of $f$?