Tweak the Learning Rate
Explore how adjusting the learning rate influences the training performance of a neural network on the MNIST dataset. Learn to identify the optimal learning rate by experimenting with different values and analyzing their impact on model accuracy.
We'll cover the following...
Adjust the learning rate
A 95% performance score on the MNIST dataset with our first neural network, using only simple ideas and simple Python, is not bad at all. If we wanted to stop here, it would be entirely justified.
But let’s see if we can make some easy improvements.
The first improvement we can try is to adjust the learning rate. Previously, we set it at , without really experimenting with different values.
Let’s try doubling it to , to see if a boost will be helpful or harmful to the overall network learning. If we run the code, we get a performance score of ...