Creating a PyTorch Pipeline

We will build a PyTorch pipeline in this lesson. We will use the CIFAR dataset as an example for this problem. The process is similar to building a TensorFlow model except for a few configuration variations. We will skip the nondistributed version and jump to creating a distributed version.

We will use three steps in the pipeline:

1. Loading the data

2. Training the model

3. Building the pipeline

Loading the data

The datasets for building deep learning models are typically very large. We could either download them from an HTTP server or from Azure Storage. In the example below, we will download the CIFAR dataset from Azure. We will use the ComputeHigh compute with more than one instance.