How neural networks work#
A neural network can receive unstructured data sets, classify data points, recognize patterns, and develop an internal representation through which it makes predictions about similar data sets.
Like humans, a neural network learns to perfect its craft over time. It goes through several iterations of computations and adjustments until it makes predictions to a reasonable accuracy.
Some of the key computational components in neural networks include:
Activation functions: Each perceptron has an activation function that standardizes its output and prevents different units from collapsing. A common activation function is the sigmoid activation function. Other activation functions are the rectified linear unit (ReLU), leaky ReLU, and tanh.
Weight: A value assigned to connections between perceptrons, estimated by the learning algorithm.
A neural network’s training process looks like this:
Receives input data: Input data is received through the input layer and passed on to hidden layer(s)
Generates outputs: The neural network usually does its initial computations by using random numbers as weight assignments
Compares outputs: The error between the generated output and required output is represented through a loss function.
Optimizes: An optimization algorithm is used to reduce the loss, an iterative process that repeats until the loss is minimized to a reasonably small value.
Our goal when training neural networks is to reduce the error or loss, which means that the network’s generated outputs will ideally match the required outputs. There are several types of loss functions, a common one being the cross-entropy loss function, which is typical in classification tasks.
To reduce the loss, we update the weights. At this stage, we don’t use random numbers as our weight assignments. Instead, we use optimization algorithms to determine the changes we need to make.
There are many optimization algorithms used to train neural networks. A popular one is the gradient descent algorithm. Gradient descent is an iterative optimization algorithm.
Data Preparation, Preprocessing, and Batching#
Before building any neural network, data preparation is the most important step.
Good preprocessing ensures faster convergence, higher accuracy, and more reliable results.
1. Load and split your dataset#
Start by dividing your data into training, validation, and test sets — typically 70/15/15 or similar ratios.