A Three-Layer Example: Working on the Hidden Layer

Output of the hidden layer

Let’s visualize the combined moderated inputs into the second hidden layer.

So far, so good, but there’s more to do. We remember those nodes apply a sigmoid activation function to make the response to the signal more like those found in nature. So let’s do that:

$$O_\text{hidden} = \text{sigmoid}(X_\text{hidden})$$

The sigmoid function is applied to each element in $X_\text{hidden}$ to produce the matrix, which has the output of the middle hidden layer:

$$O_{\text{hidden}}=\text{sigmoid}\begin{array}{c}[\begin{array}{c}1.16\\ 0.42\\ 0.62\end{array}\end{array}$$ ...