Neural Networks and Deep Learning

Biological and artificial neurons

The brains and nervous systems of humans and animals are comprised of nerve cells/neurons. These cells receive sensory input from the outside, process it, and transfer the input signals to send motor commands to the muscles. A biological neural network consists of billions of interconnected neurons that process sensory information.

Inspired by the information processing in biological neurons, scientists have developed models of artificial neurons. The following figure shows one such model.

Weights $[w_1 \ w_2\ ... \ w_n]$ are multiplied to vectorized input $[x_1 \ x_2\ ... \ x_n],$ and a bias term, $b$ , is added to the sum of weighted input. Finally, an activation function, such as a sigmoid, is applied to this input to get the output.

Letting $w_0 = b$ ...