Optimisation Function

We can also refine the technique that uses the back propagated gradients to update the network weights.

Stochastic gradient descent

Previously we used a fairly simple method, stochastic gradient descent. The method is popular because it is simple and fairly lightweight in terms of computer resources.