Prepare the MNIST Training Data

Data preprocessing

We’ve learned how to get data out of the MNIST data files and disentangle it so we can make sense of it and visualize it. We want to train our neural network with this data, but we need to think about preparing this data before we throw it at our neural network.

We saw earlier that neural networks work better if the input data and the output values are of the right shape, so that they stay within the comfort zone of the network node activation functions.

Rescaling

First we need to rescale the input color values from the larger range $0–255$ to the much smaller range $0.01–1.0$. We’ve deliberately chosen $0.01$ as the lower end of the range to avoid the problems we saw earlier with zero valued inputs, because they can artificially kill weight updates. We don’t have to choose $0.99$ for the upper end of the input, because we don’t need to avoid $1.0$ for the inputs. It’s only for the outputs that we should avoid the impossible to reach $1.0$.

Dividing the raw inputs, which are in the range $0–255$ by $255$, will bring them into the range $0-1$. Then we need to multiply by $0.99$ to bring them into the range $0.0–0.99$. Next we add $0.01$ to shift them up to the desired range $0.01–1.00$. The following Python code shows this in action: