Search⌘ K
AI Features
Home
Courses
Mastering Self-Supervised Algorithms for Learning without Labels

Contrastive Learning: The SimCLR Algorithm

Explore the SimCLR algorithm, a key contrastive learning method that trains neural networks to pull similar image embeddings closer and push dissimilar ones apart. Understand how data augmentations create positive pairs within training batches and implement contrastive loss to optimize embeddings. Gain practical coding experience applying SimCLR to unlabeled image datasets for self-supervised learning.

We'll cover the following...

What is contrastive learning?

The objective of contrastive learning is to learn neural network embeddings such that embeddings from related images should be closer than embeddings from unrelated or dissimilar images. So, given an image XaX_a, we can call it an “anchor image” and define two terms:

  • Positives: Images that are closely related to the anchor image, XaX_a. Let’s represent them by XpX_p

  • Negatives: Images that are unrelated or dissimilar to XaX_a. Let’s call them XnX_n.

The contrastive learning objective thus learns a neural network f(.)f(.) such that:

Here, sim(. , .)\text{sim}(. \ ,\ .) is any similarity measure (e.g., cosine similarity). Therefore, we need to maximize the similarity between positive pairs and minimize the similarity between negative pairs. This is illustrated in the figure below.

Contrastive learning
Contrastive learning

Contrastive loss

To enforce the contrastive property, we can define a contrastive loss function as follows:

Here, fa=f(Xa)f_a = f(X_a), fp=f(Xp) ...