Search⌘ K
Home
Courses
Introduction to Deep Learning & Neural Networks

Neural Networks

Learn about the core components of neural networks such as neurons, layers, activation functions, and the role of non-linearity in modeling complex data. Understand how to construct multi-layer perceptrons using PyTorch and grasp the universal approximation theorem. Discover how deep neural networks transform input data into feature spaces that simplify classification and improve learning outcomes.

We'll cover the following...

What is a neuron?

In the world of Neural Networks (NN), the basic building block is the neuron. NNs are nothing more than a collection of neurons organized in layers, with information passing from one layer to the other. So to understand NN, we first need to understand the neuron: the basic computing unit.

Mathematically, we have:

y=w1∗x1+w2∗x2+w3∗x3y = w_1*x_1 +w_2*x_2+ w_3*x_3

A neuron is simply a linear classifier with a single output.

import torch.nn as nn

neuron = nn.Linear(3,1, bias=False)

Looks familiar? In most applications, we also add a bias bb to shift the position of the boundary line that separates the data points. This is the infamous Perceptron.

To extend this idea, we also pass this weighted average through a non-linear function σ\sigma that will give us the decision boundary.

Why?

Because with non-linear functions between linear layers, we can model much more complex representations with less linear layers.

Non-linearities is a key component that makes NN very rich function approximators.

Putting it all together, we have:

y=σ(w1∗x1+w2∗x2+w3∗x3+b)=σ(w∗x+b)y =\sigma( w_1*x_1 +w_2*x_2+ w_3*x_3+b) = \sigma(w*x+b) ...