This module explains how to build up a basic neural network step-by-step. It also uses the MNIST dataset for detecting numbers from handwriting.

Neural Networks: Beginners to Advanced

Machine learning is one of the fastest growing fields. This module aims to teach one of the fundamental concepts of machine learning, that is neural networks. You will learn the basic concepts of building a model as well as the mathematical explanation behind the neural network. You will learn to build the neural network from scratch, in Python. You will also learn how to train and optimize your network to achieve a better result. We’ve specifically designed this module for beginners so it doesn’t require any prior programming experience. Happy learning!

# Input to the output layer 
Let's calculate the combined moderated input into the final layer $X = W \cdot I$.

The inputs into this layer are the outputs from the second layer we just worked out, named $O_\text{hidden}$. The weights are those for the links between the second and third layers, named $W_\text{hidden\_output}$, not those we just used between the first and second. Therefore, we have:

$$
X_{\text{output}} = W_{\text{hidden\_output}} \cdot O_{\text{hidden}}

$$

So, working this out in the same way we did before gives the following result for the combined moderated inputs into the final output layer:

$$
X_{\text{output}} =\begin{bmatrix}
0.3 & 0.7 & 0.5 \\
0.6 & 0.5 & 0.2\\
0.8 & 0.1 & 0.9
\end{bmatrix}
\cdot \begin{bmatrix}
0.761 \\
0.603 \\
0.650
\end{bmatrix}
$$

$$
X_{\text{output}} = \begin{bmatrix}
0.975 \\
0.888 \\
1.254
\end{bmatrix}
$$

Learn how to calculate the output of the final layer.

Prologue

Part 1 - A Little Background

Part 2 - Let's Get Started!

Part 3 - Backward Propagation of Error

Part 4 - Adjusting the Link Weights

Part 5 - A Gentle Start with Python

Part 6 - Neural Network with Python

Part 7 - Testing Neural Network against MNIST Dataset

Part 8 - Some Suggested Improvements

Part 9 - Even More Fun!

Appendix: A Small Guide to Calculus

Conclusion

A Three-Layer Example: Working on the Output Layer

Input to the output layer