Generalized Linear Regression

Single target

Consider a regression dataset $D=\{(\bold x_1,y_1),(\bold x_2,y_2),\dots,(\bold x_n,y_n)\}$, where $\bold x_i \in \R^d$ and $y_i \in \R$. A function $f_\bold w(\bold x) = \bold w^T\phi(\bold x)$ is a generalized linear model for regression for any given mapping $\phi$ of the input features $\bold x$.

The optimal parameters $\bold w^*$ can be determined by minimizing a regularised squared loss as follows:

$$\bold w^*=\argmin_{\bold w}\bigg\{\sum_{i=1}^n (\bold w^T\phi(\bold x_i)-y_i)^2 + \lambda \bold w^T\bold w\bigg\}$$

Here, $\sum_{i=1}^n (\bold w^T\phi(\bold x_i)-y_i)^2 + \lambda \bold w^T\bold w$ is the loss function denoted by $L(\bold w)$.

Suppose we want to predict the price of a house based on its size, number of bedrooms, and age. We can use a generalized linear regression to model the relationship between these input features and the target variable (the house price). The model can be defined as:

$f_{\bold w}(\bold x) = w_1 x_{1} + w_2 x_{2} + w_3 x_{3} + w_0$

Here, $x_{1}$ ...