Backpropagation Algorithm

Take a look at the mathematics of the backpropagation algorithm.

Neural Networks (NN) are non-linear classifiers that can be formulated as a series of matrix multiplications. Just like linear classifiers, they can be trained using the same principles we followed before, namely the gradient descent algorithm. The difficulty arises in computing the gradients.

But first things first.

Let’s start with a straightforward example of a two-layered NN, with each layer containing just one neuron.


  • The superscript defines the layer that we are in.
  • oLo^L denotes the activation of layer L.
  • wLw^L is a scalar weight of the layer L.
  • bLb^L is the bias term of layer L.
  • CC is the cost function, tt is our target class, and ff is the activation function.

Forward pass

Our lovely model would look something like this in a simple sketch:

Create a free account to view this lesson.

By signing up, you agree to Educative's Terms of Service and Privacy Policy