Search⌘ K
AI Features
Home
Courses
Optimization for Machine Learning with NumPy and SciPy

Higher-Order Gradients: Hessian

Explore higher-order gradients with a focus on the Hessian matrix and its role in optimization. Understand the computation of second-order partial derivatives for multivariate functions and how to implement the Hessian matrix using Python libraries NumPy and SciPy.

We'll cover the following...

Higher-order derivatives

Gradients and derivatives are also known as first-order derivatives. Algorithms like Newton’s method require computing higher-order derivatives, such as second-order derivatives. Given a multivariate function f(x1,x2,...,xm):Rm→Rf(x_1, x_2, ..., x_m): \R^m \rightarrow \R, higher-order gradients/derivatives are represented as follows:

  • ∂2f∂xi2\frac{\partial^2 f}{\partial x_i^2} ...