Search⌘ K
AI Features
Home
Courses
Fundamentals of Machine Learning: A Pythonic Introduction

Kernel Linear Discriminant

Explore the concept of kernel linear discriminant analysis, which extends traditional linear discriminants to nonlinear classification by using kernel methods. Understand how to implement these models with generalized linear regression, visualize decision boundaries, and apply one-hot encoding for multiclass problems in Python.

We'll cover the following...

In the previous lessons, we explored kernels, the kernel trick, and the Gram matrix.

We saw how kernels enable us to model nonlinear patterns by implicitly mapping data into a higher-dimensional feature space without explicitly computing that space. In this lesson, we extend those ideas to discriminant analysis.

Kernel linear discriminant models build decision boundaries in the feature space induced by a kernel function.

This enables us to construct nonlinear classification boundaries in the original input space while maintaining a linear model in the feature space. We begin with the traditional discriminant function, introduce its generalized linear form using feature mappings, develop the two-class kernel formulation, and then extend the approach to multiclass settings using one-hot targets.

Finally, we implement the method and visualize the resulting decision boundaries.

Discriminant function

Discriminant analysis aims to classify observations into one of kk classes using predictor variables. Formally, consider a dataset: D={(x1,y1),(x2,y2),…,(xn,yn)}D=\{(\bold x_1, y_1), (\bold x_2, y_2), \dots,(\bold x_n, y_n)\}, where xi∈Rd\bold x_i \in \R^d and yi∈{1,2,…,k}y_i \in \{1, 2, \dots, k\}.

A discriminant function assigns any new input xx to one of the kk classes. In linear discriminant analysis, this function is typically linear in xx, but kernel methods allow us to generalize this idea.

Generalized linear discriminant function

A generalized linear discriminant is a discriminant function that is linear not in the original input xx, but in the transformed feature space ϕ(x)ϕ(x). This replaces the standard linear form wTxw^{T}x with wTϕ(x ...