Prepare Data: Inputs and Outputs

An example of a weight update

Not all attempts at using neural networks will work well. Some of the reasons they don’t work well can be addressed by thinking about the training data and the initial weights, and designing a good output scheme. Let’s look at each of these in turn.

Prepare inputs

Let’s have a look at the diagram of the sigmoid activation function below. We can see that if the inputs are large, the activation function gets very flat.