Mastering Self-Supervised Algorithms for Learning without Labels

Advanced

31 Lessons

Certificate of Completion

AI-POWERED

Explanations

This course includes

41 Playgrounds

110 Illustrations

This course includes

41 Playgrounds

110 Illustrations

Course Overview

This course covers self-supervised algorithms, which are useful for large pools of unlabelled data or when obtaining a high-quality labeled dataset is difficult. These algorithms leverage the supervisory signals from the structure of the unlabeled data to predict any unobserved or hidden property of the input. You’ll start with the fundamentals of self-supervised learning and then implement your first class of algorithms. You’ll learn to generate pseudo labels and use these labels for training models using...Show More

What You'll Learn

An understanding of self-supervised learning and its advantage over unsupervised learning

Working knowledge of designing your self-supervised learning tasks/objectives

Hands-on experience implementing and modifying existing self-supervised learning objectives to learn from unlabelled data

Ability to transfer and evaluate your self-supervised network representations on a downstream task

Familiarity with core components of self-supervised learning, including pretext tasks, similarity maximization, redundancy reduction, and masked image modeling

Course Content

Introduction to Self-Supervised Learning

Course Overview and Prerequisites

Introduction to Self-Supervised Learning Introductory Quiz on Self-Supervised Learning

Pretext Tasks

Introduction to Pretext Tasks Predicting Relative Position of Patches Predicting Rotation Solving Jigsaw Puzzles Quiz on Pretext Tasks Coding Exercise on Pretext Tasks

Similarity Maximization and Redundancy Reduction

Introduction to Similarity Maximization Contrastive Learning: The SimCLR Algorithm SimCLR Training Objective The MoCo-v2 Algorithm Clustering: The DeepCluster Algorithm DeepCluster Training Distillation: The BYOL Algorithm BYOL Training The SimSiam Algorithm Barlow Twins Coding Exercise on Barlow Twins Quiz on Similarity Maximization and Redundancy Reduction

Masked Image Modeling

Introduction to Masked Image Modeling Simple Masked Image Modeling Masked Autoencoders: Masking and Encoder Masked Autoencoders: Decoder and Loss Function Masked Siamese Networks: Masking Strategy and Encoder Masked Siamese Networks: Objective and Training Coding Exercise on Masked Modeling Quiz Conclusion

Appendix

References for Research Papers

COURSE AUTHOR

How You'll Learn

You don’t get better at swimming by watching others. Coding is no different. Practice as you learn with live code environments inside your browser.

Videos are holding you back. Educative‘s interactive, text-based lessons accelerate learning — no setup, downloads, or alt-tabbing required.

Learn faster and smarter with adaptive AI tools embedded in every Educative course.

Built-in assessments let you test your skills. Completion certificates let you show them off.

FOR TEAMS

Interested in this course for your business or team?

Unlock this course (and 1,000+ more) for your entire org with DevPath