Backpropagation: Splitting the Error

The output error backpropagation

Neural networks learn by refining their link weights. Therefore, to backpropagate the error to internal nodes, we split the output layer errors in proportion to the size of the connected link weights, and then recombine these bits at each internal node. The following diagram shows a simple neural network with three layers: an input layer, a hidden layer, and the final output layer.

Working back from the final output layer at the right-hand side, we can see that we use the errors in that output layer to guide the refinement of the link weights feeding into the final layer. We’ve labeled the output errors more generically as $e_\text{output}$, and weights of the links between the hidden and output layer as $w_{ho}$ ...