Implementation of autoencoders using TensorFlow

Imagine a magical device that can take a picture of an object and then magically compress that picture into a tiny, hidden code. Later, we can recreate the original picture almost perfectly using this code. This magical device is an autoencoder in the world of computers and artificial intelligence.

Autoencoder

An Autoencoder is a fundamental neural network architecture used in generative AI. It plays a crucial role in generating data or features that resemble the input data it was trained on. An autoencoder is designed to encode and decode data, typically used for dimensionality reduction, feature learning, and generative tasks.

Components

Here are the two main components of an autoencoder:

Encoder: The encoder takes an input data point and transforms it into a compressed representation, often called a latent space or encoding. This encoding typically has lower dimensionality than the input data and captures essential features or patterns.
Decoder: The decoder takes the encoded representation and attempts to reconstruct the original input data. The goal is to produce an output that closely resembles the input data, effectively learning a mapping from the lower-dimensional encoding space back to the original data space.

Implementation

Let’s see the implementation of an autoencoder using TensorFlow and Keras. In the Encoder class, the input layer has several neurons specified by the input_dim (64 in this case), followed by a hidden layer with the hidden_dim neurons (32 in this case), which results in a reduction in the number of neurons from 64 to 32. The output layer then has output_dim neurons (8 in this case). In contrast, in the Decoder class, the input layer has output_dim neurons (8 in this case), followed by a hidden layer with hidden_dim neurons (32 in this case), and an output layer with input_dim neurons (64 in this case), leading to an increase in neurons. The activation function used in both the encoder and decoder layers is the rectified linear unit (ReLU), which is specified by the activation='relu' parameter in the Dense layer definitions.

import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Model
# Define the Encoder class
class Encoder(tf.keras.Model):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super(Encoder, self).__init__()
        self.input_layer = Dense(input_dim, activation='relu')
        self.hidden_layer = Dense(hidden_dim, activation='relu')
        self.output_layer = Dense(output_dim, activation='relu')
    def call(self, x):
        x = self.input_layer(x)
        x = self.hidden_layer(x)
        x = self.output_layer(x)
        return x
# Define the Decoder class
class Decoder(tf.keras.Model):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super(Decoder, self).__init__()
        self.input_layer = Dense(input_dim, activation='relu')
        self.hidden_layer = Dense(hidden_dim, activation='relu')
        self.output_layer = Dense(output_dim, activation='relu')
    def call(self, x):
        x = self.input_layer(x)
        x = self.hidden_layer(x)
        x = self.output_layer(x)
        return x
# Define the Autoencoder class
class Autoencoder(tf.keras.Model):
    def __init__(self, encoder_dim, decoder_dim):
        super(Autoencoder, self).__init__()
        self.encoder = Encoder(*encoder_dim)
        self.decoder = Decoder(*decoder_dim)
    def call(self, x):
        x = self.encoder(x)
        x = self.decoder(x)
        return x
# Define the dimensions for the encoder and decoder
input_dim = 64
hidden_dim = 32
output_dim = 8
encoder_dim = (input_dim, hidden_dim, output_dim)
decoder_dim = (output_dim, hidden_dim, input_dim)
# Create an instance of the Autoencoder model
model = Autoencoder(encoder_dim, decoder_dim)
# You can adjust the batch size as needed
batch_size = 1
input_shape = (batch_size, input_dim)
# Print summary of each layer individually
for layer in model.layers:
    print(layer.name)
    layer.build(input_shape)
    layer.summary()
    print("\n")

Code explanation

Lines 1–3: We import the necessary TensorFlow and Keras modules to build and define neural network models.
Lines 6–17: We define an Encoder class with three layers: input, hidden, and output. It takes the input data of the dimension— input_dim—processes it through the layers with the ReLU activation functions, and produces an output of the dimension— output_dim—when called with the input x.

Note: This class is typically used as part of a larger neural network architecture for feature encoding or dimension reduction tasks.

Lines 20–31: We define a neural network model called Decoder, which has three layers with the ReLU activation: an input layer, a hidden layer, and an output layer. The call method specifies the forward pass, where the input x is passed through these layers sequentially, and the result is returned as the output.
Lines 34–43: We define an Autoencoder class using TensorFlow’s Keras API. It has an encoder and a decoder, both specified with the encoder_dim and decoder_dim dimensions. When we call this autoencoder with the x input data, it first encodes the data using the encoder and then decodes it using the decoder, returning the reconstructed output x.
Lines 46–64: We define the dimensions for the encoder and decoder in an autoencoder neural network, create an instance of the autoencoder model using these dimensions, and then print a summary of each layer in the model, including its name, build information, and a summary of its architecture. The batch size and input shape are also specified.

Applications

Let’s explore the diverse applications of autoencoders and demonstrate their versatility in various domains:

Dimensionality reduction: Autoencoders can reduce the dimensionality of data while preserving its essential features, making it useful for efficient storage and transmission.
Image denoising: Autoencoders can be trained to remove noise from images. By feeding noisy images as input and clean images as target outputs, the autoencoder learns to denoise images effectively. This is used in image enhancement and restoration tasks.
Image colorization: Autoencoders can help colorize black-and-white images by predicting the color information based on the provided grayscale input.

Unlock your potential: Tensorflow series, all in one place!

To continue your exploration of Tensorflow, check out our series of Answers below:

Implementation of Autoencoder using Tensorflow
Learn how autoencoders efficiently encode and decode data, which is crucial in tasks like dimensionality reduction, denoising, and colorization.
What is TensorFlow object detection model
Learn how TensorFlow's object detection API provides tools for creating and deploying models, featuring pretrained models, customizable training, and diverse application use cases.
PyTorch vs. Tensorflow
Learn how PyTorch is ideal for ease of use and rapid prototyping, while TensorFlow excels in production deployment and scalability for large-scale projects.
How to shuffle a dataset in TensorFlow?
Learn how to use TensorFlow's shuffle() method to introduce randomness in datasets, ensuring models don't learn unintended sample patterns.

Free Resources