Polynomial Regression

Why polynomial regression?

Until now, we’ve assumed a linear relationship between features and targets. What if the assumption of linearity doesn’t hold even approximately? The figure below shows that the relationship between a single feature, $x$ and a single target, $y$ isn’t linear in the data set.

$$f(x) = \hat{y} = \sum_{i=0}^d w_i x^i$$

Assuming degree to be known

Let’s look at how we can estimate the parameters, $\bold{w}$, if we assume, for the sake of simplicity, that the degree of the polynomial to be fit is known.

Example of polynomial with degree 3

Let’s take a simple example with only one feature, $x$, and a single target, $y$. Let’s also assume that $y_i = w_0+w_1x_i+w_2x_i^2+w_3x_i^3$ ...