Search⌘ K
AI Features
Home
Courses
An Introductory Guide to Data Science and Machine Learning

Backpropagation: Part 1

Learn the fundamentals of backpropagation, a process that updates neural network weights to reduce errors during training. Understand key formulas, error propagation through layers, and gradient descent variants. This lesson builds the foundation needed to train and improve neural networks effectively in deep learning.

We'll cover the following...

Backpropagation

Backpropagation involves the process of updating the neural network weights while training the neural network on the training dataset, and it updates the weights in such a way that the total error for the problem under consideration is minimized.

What is the backward pass?

It involves propagating the error (calculated at the output layer) backwards to the layers and updating the weights.

General context of a neuron and formulas

If we look into the general context of a neuron from the diagram above, we get the following equations:

  • aj=∑iwijzia_j=\sum_{i}w_{ij}z_{i}: This sums up all the inputs multiplied by the respective weights.

  • zj=σ(aj)=11+e−ajz_j=\sigma{(a_j)}=\frac{1}{1+e^{-a_j}} This applied the activation function on the input being passed in.

  • Error=12∑instances∑k(yk−tk)2Error=\frac{1}{2}\sum_{instances}\sum_{k}(y_k-t_k)^2 This computes the error on all the training instances and across all the output nodes.

  • ...