Weight Updates Calculated

An example of calculating a weight update

The following network is one we’ve worked with before, but this time we’ve added example output values from the first hidden node $o_{j=1}$ and the second hidden node $o_{j=2}$. These are just made-up numbers to illustrate the method and aren’t properly worked out by feeding signals forward from the input layer.

We want to update the weight $w_{11}$ between the hidden and output layers, which currently has the value $2.0$.

Let’s write out the error slope again:

$$\frac{\partial E}{\partial w_{jk}} = -(t_k - o_k)\cdot \text{sigmoid}\left(\sum_j w_{jk} \cdot o_j \right) \left(1-\text{sigmoid}\left(\sum_j w_{jk} \cdot o_j\right)\right)\cdot o_j$$ ...