Linear Classifiers

What is a linear classifier?

Suppose we want to build a machine learning model to classify the following points into two categories based on their color. It is very easy to see that we can find a single point that separates them perfectly. The goal of our model is to find this point.

The easiest way to do that is to build a linear classifier. Our classifier has the form $f(x, w) = w_1*x_1+w_2$. The purpose of $f(x,w)$ will be to find the parameters $w_1$ and $w_2$, so that any corresponding scalar point (1D) can be distinguished perfectly. If $f(x,w)>0$, the point belongs to the blue category. Otherwise, it belongs to the red. Sounds easy?

Let’s extend this idea to 2D data points!

Each point will now be represented as $(x_1, x_2)$.

For the 2D case, we need to find a line (instead of a point) that separates our 2D points, so our classifier will be $f(x,w) = w_1*x_1 +w_2*x_2+ w_3$ ...