Search⌘ K
AI Features
Home
Courses
Fundamentals of Machine Learning: A Pythonic Introduction

Kernels

Explore kernel methods in machine learning to understand how input data is transformed into higher-dimensional spaces for easier classification and regression. Learn about the kernel trick that allows efficient computation without explicit feature mapping, and become familiar with popular kernels like linear, polynomial, and RBF. Understand the role of the Gram matrix in kernel-based algorithms and see practical Python implementations.

We'll cover the following...

Kernels are an important concept in machine learning and pattern recognition. They’re a mathematical function that maps input data into a high-dimensional feature space where it’s easier to classify or analyze. Kernels allow us to perform complex computations on data that would otherwise be difficult or impossible to process in its original form.

Applying kernel on a 2D dataset
Applying kernel on a 2D dataset

Kernel function

A kernel function can be thought of as a dot product in the feature space defined by the mapping ϕ\phi. Given two input vectors xi\bold x_i and xj\bold x_j, the dot product in the feature space can be represented as ϕ(xi)Tϕ(xj)\phi(\bold x_i)^T\phi(\bold x_j). It’s possible to compute the dot product ϕ(xi)Tϕ(xj)\phi(\bold x_i)^T\phi(\bold x_j) in the feature space via a kernel function k(xi,xj)k(\bold x_i, \bold x_j) on the input vectors.

Polynomial kernel example

Assume two input vectors xi=[x1ix2i]T\bold x_i=\begin{bmatrix}x_{1i} & x_{2i}\end{bmatrix}^T and xj=[x1jx2j]T\bold x_j=\begin{bmatrix}x_{1j} & x_{2j}\end{bmatrix}^T ...