Create a Target Array

Construction of a target array

Now we need to turn these ideas into target arrays for the neural network training. We can see that if the label for a training example is $5$, we need to create a target array for the output node where all the elements are small except the one corresponding to the label $5$. That could look like:

$$[0, 0, 0, 0, 0, 1, 0, 0, 0, 0]$$

In fact, we need to rescale those numbers because we’ve already seen how trying to get the neural network to create outputs $0$ and $1$, which are impossible for the activation function, will drive large weights and a saturated network. So, we’ll use the values $0.01$ and $0.99$ instead, so the target for the label $5$ should be:

$$[0.01, 0.01, 0.01, 0.01, 0.01, 0.99, 0.01, 0.01, 0.01, 0.01]$$

Let’s have a look at the following Python code to construct the target matrix: