Gain insights into solving optimization problems in Python. Delve into essential mathematical tools, metaheuristic methods, and techniques for constrained optimization. Become proficient in efficient Python solutions.

Optimization with Python from zero to hero.png

cvxpy.tar.gz

cvxpy

Optimization theory seeks the best solution, which is pivotal for machine learning, cost-cutting in manufacturing, refining logistics, and boosting finance profits.

This course provides a detailed description of different optimization problems and the techniques used to solve them. You’ll begin with the formal definition of an optimization problem and an overview of essential mathematical tools: derivatives, gradients, and Hessian. With this knowledge, you’ll implement solutions for several optimization problems. You’ll also learn approximate metaheuristic population methods like particle swarm optimization and genetic algorithms, and solve constrained optimization problems using techniques like penalty methods and constraint simplification. Finally, you’ll solve linear programs and integer programs. 

By the end of this course, you’ll become proficient in formulating different kinds of problems as optimization problems and become skilled in the Python tools that efficiently solve these problems.

Mastering Optimization with Python

We need to solve:

$$
\max_{x,y} -x^2 - y^2 + 6x + 4y
$$

We already know how to solve this problem with other algorithms, but let's practice what we've learned. Using the gradient and the Hessian, for example, we can solve this problem analytically. The maximum of this function is $(3, 2)$.

This time, a solution will be a vector $(x, y)$. For the genetic algorithm implementation, we use the following operations:

* `mutation`: We randomly sum one or subtract one from the first component of the vector.
* `crossover`: We make a new vector with the first component of the first parent and the second component of the second parent.
* `score`: If $f(x) < 0$, then the score of $x$ will be $|f(x)| * 10^{-9}$. Otherwise, it'll be $f(x)$. As we know there are solutions $s$ so that $f(s) > 0$, we know that any value of $f$ that's less than $0$ isn't the solution we're looking for. But the score should always be positive, so we need to make sure any negative solution is worse than any positive one, which is why we add the $10^{-9}$ factor.

Let's see the genetic algorithm implementation.

Introduction

Derivatives and Gradients

First Optimization Algorithms

Population Methods

Adding Constraints

Linear Constrained Optimization

Summary and Conclusion

Appendix

Solution to Exercise 1: Using Both Algorithms