Search⌘ K
AI Features
Home
Courses
Fundamentals of Machine Learning: A Pythonic Introduction

Implementation of a Neural Network

Explore how to implement a neural network using PyTorch to perform binary classification. This lesson guides you through building and training models on both numeric and custom image datasets, illustrating forward and backward passes, loss functions, and optimization methods. By the end, you'll understand how to create flexible neural network classifiers and evaluate their performance effectively.

We'll cover the following...

We mastered the theoretical foundation of neural networks (NNs), understanding their structure, the role of non-linear activations, and the mathematics of the forward and backward passes.

However, implementing these complex calculations from scratch in Python can be cumbersome. This lesson bridges that gap by introducing a modern, high-level framework: PyTorch.

Introduction to PyTorch

PyTorch is an open-source deep learning framework, quite popular among researchers, developers, and data scientists due to its simplicity, flexibility, and efficiency in building and training deep neural networks. It’s developed and maintained by Facebook’s AI Research Lab (FAIR). PyTorch provides an intuitive and Pythonic interface that enables users to easily construct complex computational graphs for deep learning models.

Binary classification for numerical dataset

An example of implementing a simple neural network using PyTorch is given below. In this example, we’ll create a neural network with one hidden layer for a binary classification task. We’ll use the famous Iris dataset and aim to classify whether a given flower belongs to the Iris setosa species or not.

Note: The slight variation in the plots, each time the code is run, can be attributed to the random initialization of the neural network weights and biases, as well as the randomness introduced during the train-test split. This can lead to different initial conditions and data splits, that result in slightly different training trajectories and test performance.

Python 3.10.4
import torch
import torch.nn as nn
import torch.optim as optim
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
# Load Iris dataset
data = load_iris()
X, y = data.data, data.target
X = X[y != 2]  # Consider only two classes: Iris Setosa (0) and Iris Versicolor (1)
y = y[y != 2]
# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Normalize features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# Convert NumPy arrays to PyTorch tensors
X_train = torch.tensor(X_train, dtype=torch.float32)
y_train = torch.tensor(y_train, dtype=torch.float32)
X_test = torch.tensor(X_test, dtype=torch.float32)
y_test = torch.tensor(y_test, dtype=torch.float32)
# Define the simple neural network model
class SimpleNeuralNetwork(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(SimpleNeuralNetwork, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, output_size)
        self.sigmoid = nn.Sigmoid()
    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.sigmoid(out)
        return out
# Set the model parameters
input_size = X_train.shape[1]
hidden_size = 5
output_size = 1
# Initialize the model
model = SimpleNeuralNetwork(input_size, hidden_size, output_size)
# Define the loss function and optimizer
criterion = nn.BCELoss()  # Binary Cross Entropy Loss
optimizer = optim.SGD(model.parameters(), lr=0.01)
# Lists to store training and test losses
train_losses = []
test_losses = []
train_accuracies = []
test_accuracies = []
# Training the model
num_epochs = 1000
for epoch in range(num_epochs):
    # Forward pass
    outputs = model(X_train)
    loss = criterion(outputs, y_train.view(-1, 1))
    # Backward and optimize
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    if (epoch + 1) % 100 == 0:
        print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')
    
    train_losses.append(loss.item())
    
    # Compute training accuracy
    predicted_train = (outputs >= 0.5).squeeze().float()
    train_accuracy = (predicted_train == y_train).sum().item() / y_train.size(0)
    train_accuracies.append(train_accuracy)
    
    # Evaluate the model on the test set
    with torch.no_grad():
        model.eval()
        test_outputs = model(X_test)
        test_loss = criterion(test_outputs, y_test.view(-1, 1))
        test_losses.append(test_loss.item())
        
        # Compute test accuracy
        predicted_test = (test_outputs >= 0.5).squeeze().float()
        test_accuracy = (predicted_test == y_test).sum().item() / y_test.size(0)
        test_accuracies.append(test_accuracy)
# Plot the training and test loss curves
plt.figure(figsize=(12, 5))
plt.subplot(1, 2, 1)
plt.plot(range(1, num_epochs+1), train_losses, label='Training Loss', color='blue')
plt.plot(range(1, num_epochs+1), test_losses, label='Test Loss', color='orange')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.grid(True)
plt.title('Training and Test Loss Curves')
# Plot the training and test accuracy curves
plt.subplot(1, 2, 2)
plt.plot(range(1, num_epochs+1), train_accuracies, label='Training Accuracy', color='blue')
plt.plot(range(1, num_epochs+1), test_accuracies, label='Test Accuracy', color='orange')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.grid(True)
plt.title('Training and Test Accuracy Curves')
# Evaluate the model on the test set
with torch.no_grad():
    model.eval()
    test_outputs = model(X_test)
    predicted = (test_outputs >= 0.5).squeeze().float()
    accuracy = (predicted == y_test).sum().item() / y_test.size(0)
    print(f'Test Accuracy: {accuracy:.2f}')

Here is the explanation for the code above:

  • Lines 1–6: We import necessary libraries such as torch, nn, optim, load_iris, train_test_split, and StandardScaler for data preprocessing. We also use matplotlib.pyplot to visualize the insights.

  • Lines 10–27: Data preprocessing involves loading the Iris dataset, filtering it for classes 0 and 1, splitting it into training and test sets, and converting it to PyTorch tensors, which are multi-dimensional arrays used by PyTorch for efficient computation and automatic differentiation.

  • Lines 30–48: We define the SimpleNeuralNetwork class, which inherits from nn.Module. It consists of an input layer connected to a hidden layer with a ReLU activation, followed by an output layer with a sigmoid activation. The forward method specifies the data flow through these layers. We also set the model parameters, including the input size, hidden layer size, and output ...