Linear Regression

Function approximation

Approximating a function means estimating the values of its parameters. Consider the SSE function we discussed in the previous lesson.

$$SSE(\bold{\hat w})=(A\bold{\hat w}-\bold{b})^T(A\bold{\hat w}-\bold{b})$$

Approximating SSE means estimating the vector, $\bold w$, that nearly satisfies the linear system, also called the linear least squared error solution.

Formal definition

Consider a data set, $D=\{(\bold{x_1},\bold{y_1}),(\bold{x_2},\bold{y_2}),...,(\bold{x_n},\bold{y_n})\}$, where each entry is a pair, $\bold{x_i}$ and $\bold{y_i}$, of objects (scalars, vectors, matrices, and so on). Function approximation seeks a function, $f_\bold{w}$, such that:

$$f_\bold{w}(\bold{x_i})\approx\bold{y_i}$$

Example

Let $D=\{(4,1),(-3,9)\}$. The function ...