What is the difference between triplet and contrastive loss?
In the field of computer vision, contrastive and triplet loss help to learn the embeddings that can capture the similarity or dissimilarity of the data points. In real-life scenarios, we use them for tasks like face recognition, recommendation systems, or metric learning.
In this Answer, we will cover the following topics:
Triplet loss.
Contrastive loss.
Difference between the triplet and the contrastive loss.
Triplet Loss
In triplet loss, we use three data points which are:
Anchor point
Positive point
Negative point
Before diving into the details, let us make a clear definition of positive points and negative points. A positive point is a point that is similar to the anchor point while a negative point is dissimilar to the anchor point.
The main objective to use triplet loss is to minimize the distance between an anchor point and a positive point while maximizing the distance between an anchor point and a negative point.
Mathematical Intuition
The mathematical formulation of triplet loss is:
where:
represents the distance between an anchor and a positive point. represents the distance between an anchor and a negative point. represents the bias. It tells how far the negative point must be from the anchor.
Contrastive loss
Unlike triplet loss, we use only two points here which are:
Anchor point
Comparison point
A comparison point is a point that we use to compare with the anchor point. It can be a negative or a positive point as well.
In contrastive loss, we penalize similar points by minimizing the distance between them and maximizing the dissimilar pair, if their distance is less than the margin m.
Mathematical Intuition
The mathematical formulation of triplet loss is:
is the distance between the anchor and the comparison point. represents if the pair is similar or dissimilar. is 0 for the similar pair otherwise 1. represents the separation between the similar and the dissimilar pair.
The point at the center of a circle represents the comparison point.
Difference
The differences between contrastive loss and the loss are given below:
Triplet loss | Contrastive loss | |
Use | We use triplet loss to learn different embeddings for positive, negative, and anchor data points. | We use contrastive loss to differentiate between similar and dissimilar data points. |
Input | The input of the triplet loss consists of an anchor point, a positive point, and a negative point.
| The input in contrastive loss requires a comparison point and an anchor point. |
Calculations | It maximizes the distance between an anchor point and a negative point and minimizes the distance between the positive point and an anchor point. | It encourages similar points to close together while dissimilar points to be far apart. |
Efficiency | It requires more time in training because of the large number of samples. | It is more efficient because it requires only similar and dissimilar points. |
Applications | Its applications are in face recognition and in-person re-identification. | Its applications are in similarity learning, image retrieval, and in recommendations systems |
After the clear understanding of triplet and contrastive loss given in this Answer, let's dive into a small quiz.
What is the main objective of contrastive loss and triplet loss?
Contrastive loss: Encourages similar samples to be closer and dissimilar samples to be farther apart.
Triplet loss: Encourages the anchor and positive samples to be closer than the anchor and negative samples.
Both A and B.
None of the above.
Free Resources
