Search⌘ K
AI Features
Home
Courses
Optimization for Machine Learning with NumPy and SciPy

Coordinate Descent

Explore the coordinate descent algorithm, which optimizes multivariate convex functions by sequentially minimizing along individual coordinate directions. Learn how this method compares to gradient descent, its advantages, and practical applications in regression, image, and signal processing. Gain insight into its implementation using Python libraries and visualize its convergence.

We'll cover the following...

The coordinate descent algorithm

Consider a multivariate function f(x)=f(x1,x2,...,xn)f(x) = f(x_1, x_2, ..., x_n) that we want to optimize. Using the gradient descent algorithm, the updates will happen in all directions at once.

Coordinate descent is a variation of gradient descent that tries to find the minimum of a function by minimizing it (i.e., performing gradient descent) along one coordinate direction at a time. Starting from an initial point x0x^0, coordinate descent defines ...