Search⌘ K
AI Features
Home
Courses
Fundamentals of Machine Learning: A Pythonic Introduction

Kernels

Explore how kernel functions map input data into high-dimensional feature spaces, enabling non-linear classification and regression. Understand the kernel trick that simplifies computations without explicit mapping, and learn about popular kernels such as polynomial and RBF. Discover the role of the Gram matrix in efficiently applying kernel methods across datasets.

We'll cover the following...

Kernels are an important concept in machine learning and pattern recognition. They’re a mathematical function that maps input data into a high-dimensional feature space where it’s easier to classify or analyze. Kernels allow us to perform complex computations on data that would otherwise be difficult or impossible to process in its original form.

Applying kernel on a 2D dataset
Applying kernel on a 2D dataset

Kernel function

A kernel function can be thought of as a dot product in the feature space defined by the mapping ϕ\phi. Given two input vectors xi\bold x_i and xj\bold x_j, the dot product in the feature space can be represented as ϕ(xi)Tϕ(xj)\phi(\bold x_i)^T\phi(\bold x_j). It’s possible to compute the dot product ϕ(xi)Tϕ(xj)\phi(\bold x_i)^T\phi(\bold x_j) in the feature space via a kernel function k(xi,xj)k(\bold x_i, \bold x_j) on the input vectors.

Polynomial kernel example

Assume two input vectors xi=[x1ix2i]T\bold x_i=\begin{bmatrix}x_{1i} & x_{2i}\end{bmatrix}^T and xj=[x1jx2j]T\bold x_j=\begin{bmatrix}x_{1j} & x_{2j}\end{bmatrix}^T ...