Errors in the Training Classifier

Calculate an error

Let’s look at the first training example: the width is $3.0$ and the length is $1.0$ for a ladybug. If we tested the $y = Ax$ function with this example, where $x$ is $3.0$, we would get:

$$y = (0.25) \times (3.0) = 0.75$$

The function with the parameter $A$, which is set to the randomly chosen value of $0.25$, is suggesting that for a bug of width $3.0$, the length should be $0.75$. We know that’s too small because the training data example tells us it must be a length of $1.0$. So, we have a difference, which indicates an error. Like before, with the miles to kilometers predictor, we can use this error to inform how we adjust the parameter $A$.

But before we do that, let’s think about what $y$ should be again. If $y$ is $1.0$, then the line goes right through the point where the ladybug sits at $(x,y) = (3.0, 1.0)$. It’s a subtle distinction, but we don’t actually want that. We want the line to go above that point. Why? Because we want all the ladybug points to be below the line, not on it. The line needs to be a divider between ladybugs and caterpillars, not a predictor of a bug’s length given its width.

So, let’s try to aim for $y = 1.1$ when $x = 3.0$. It’s just incrementally above $1.0$. We could’ve chosen $1.2$ or even $1.3$, but we don’t want a larger number like $10$ or $100$ because that would make it more likely that the line will go above both the ladybugs and caterpillars data points, resulting in a separator that isn’t useful at all. So the desired target is $1.1$, and the error $E$ is:

$$\text{Error = (desired target $-$ actual output)}$$

which is,

$$E = 1.1 - 0.75 = 0.35$$

Let’s pause and remind ourselves what the error, the desired target, and the calculated value mean visually.