Error Backpropagation with Matrix Multiplication

Vectorize the operations

Can we use matrix multiplication to simplify all the laborious calculation? It helped earlier when we were doing multiple calculations to feed the input signals forward.

To see if error backpropagation can be made more concise using matrix multiplication, let’s write out the steps using symbols. By the way, this is called vectorizing the process.

Being able to express a lot of calculations in matrix form makes it more concise for us to write down. It also allows computers to efficiently do all the work because they take advantage of the repetitive similarities in the calculations that need to be done.

The starting point is the errors that emerge from the neural network at the final output layer. Here, we only have two nodes in the output layer. These are $e_1$ and $e_2$:

$$\text{error}_\text{output} = \begin{bmatrix} e_1\\e_2 \end{bmatrix}$$