Assembling a Neural Network from Perceptrons

Recap

The first partFrom chapter-1 (How Machine Learning Works To chapter-8 (The Perceptron)) of this course was all about the perceptron. The second partFrom chapter-9 (Designing the Network) To chapter-15 (Let’s Do Development) explains the advanced concepts in machine learning, and the most important idea in this chapter is the neural network. Neural networks are way more powerful than perceptrons. In the Where Perceptrons Fail lesson, we learned that perceptrons need simple linearly separable data. By contrast, neural networks can deal with unorganized data, like photos of real-world objects.

Even for a simple dataset like MNIST, our perceptron was scraping by, making almost one mistake every ten characters. With neural networks, we can aim for an order of magnitude better accuracy. In this part of the chapter, we’ll build an MNIST classifier that reaches 99% accuracy, which is one error every 100 characters.

Now let’s design a neural network that classifies MNIST digits.

Assemble perceptrons

Let’s see how to build a neural network, starting with the perception that we already have. As a reminder, the perceptron is a weighted sum of the inputs, followed by a sigmoid: