SimCLR Training Objective

Now that we have two augmented versions of the input batch, $T_1(B)$ and $T_2(B)$, we'll look into other components of the SimCLR training pipeline.

Network architecture

As shown in the figure below, the two augmented versions of an image, $X_i$ (i.e., $T_1(X_i)$ and $T_2(X_i)$), are passed through the neural network $f(.)$ to get the penultimate feature representations, $h_{i1}$, and $h_{i2}$, respectively. These feature representations are passed again through a multilayer perceptron (MLP) projection head $g(.)$ to get the feature embeddings $z_{i1}$ and $z_{i2}$, respectively.