Forward and Backward Propagation in RNNs

Forward propagation

This is the process of calculating and storing the intermediate variables (including outputs) for the neural network in order from the input layer to the output layer. We will not discuss this in too much detail, as it is assumed you already have some background knowledge on this.

A simple architecture of an RNN is shown below: