Transform the Output into an Error Function
Understand how to transform neural network outputs into meaningful error functions to guide training. Explore the strengths and weaknesses of simple, absolute, and squared error methods and discover why squared error is preferred for effective gradient descent optimization.
We'll cover the following...
Error functions
The output of a neural network is a complex difficult function with many parameters and the link weights that influence its output. Can we use gradient descent to work out the right weights? Yes, as long as we pick the right error function.
The output function of a neural network isn’t an error function itself. But we know we can turn it into one easily, because the error is the difference between the target training values and the actual output values.
There’s something to watch out for here. Look at the following table of training and actual values for three output nodes, together with candidates for an error function.
Comparison of Different Error Functions
Network Output | Target Output | Error (target – actual) | Error | target – actual | | Error (target – actual)2 |
0.4 | 0.5 | 0.1 | 0.1 | 0.01 |
0.8 | 0.7 | – 0.1 | 0.1 | 0.01 |
1.0 | 1.0 | 0 | 0 | 0 |
Sum | 0 | 0.2 | 0.02 | |
Simple error
The first candidate for an error function is simply ...