Introduction to K-Nearest Neighbors

The k-nearest neighbors (KNN) is widely used for classification and is one of the simplest machine learning algorithms. The principle behind the method is to find a predefined number of training samples closest in the distance to the test point and predict the label from these. In simple words, KNN stores the entire dataset as a training algorithm and categorizes the test data point using stored information of its nearest neighbors, where $k$ is the number of neighbors. Let’s learn with example data.

Example

Let’s consider:

• We have a dataset with two classes, A and B, and the entire data, along with its features, f1, and f2, are stored as a training algorithm.

• We want to predict the class for red, green, and blue test data points.

• Based on the association with their neighboring points, it's straightforward to predict the class for red and green.