INTERACTIVE COURSE

Getting Started with Google BERT

Intermediate

120 Lessons

25h

Certificate of Completion

AI Explanations

26 Playgrounds

12 Quizzes

250 Illustrations

Takeaway Skills

An in-depth knowledge of transformer models, including the workings of encoders and decoders

A firm grasp of BERT’s core concepts, pre-training methods, and subword tokenization algorithms

The ability to effectively use pre-trained BERT models, from extracting contextual embeddings to fine-tuning for various tasks

A thorough understanding of various BERT variants, such as ALBERT, RoBERTa, ELECTRA, and SpanBERT

An understanding of BERT’s applications in multilingual contexts and domain-specific variants like ClinicalBERT and BioBERT

Course Overview

The transformer, a cutting-edge deep learning architecture, plays a pivotal role in Natural Language Processing (NLP) tasks. Leading NLP models like BERT, GPT, and T5 are based on the transformer. BERT, in particular, has revolutionized NLP by delivering state-of-the-art results. This course is your entry into Google's BERT architecture. It begins with the basics of transformers and covers encoder and decoder mechanisms. You will learn to pre-train and use BERT. Exploring various BERT variants and related ...Show More

Course Content

Before we Start

About the course

Starting Off with BERT

Introduction: BERT

A Primer on Transformers

Introduction to the Transformer Understanding the Encoder of the Transformer Self-Attention Mechanism Understanding the Self-Attention Mechanism Multi-Head Attention Mechanism Learning Position with Positional Encoding Encoder: Feedforward and Add and Norm Component Putting all the Encoder Components Together Understanding the Decoder of the Transformer Masked Multi-Head Attention Multi-Head Attention Decoder: Feedforward and Add and Norm Component Putting All the Decoder Components Together Putting the Encoder and Decoder Together Summary: A Primer on Transformers Exercise: A Primer on Transformers Exercise: Decoder Architecture Quiz: A Primer on Transformers

Understanding the BERT Model

Introduction to BERT Model Working of BERT Configurations of BERT Pre-Training the BERT Model The WordPiece Tokenizer Pre-Training Strategies for the BERT Model Masked Language Modeling Next Sentence Prediction Pre-Training Procedure Subword Tokenization Algorithms Byte Pair Encoding Byte-level Byte Pair Encoding & WordPiece Algorithms Summary: Understanding the BERT Model Quiz: Understanding the BERT Model

Getting Hands-On with BERT

Exploring the Pre-Trained BERT Model Transformers Extracting Embeddings From All Encoder Layers of BERT Fine-Tuning BERT for Downstream Tasks Sentiment Analysis Natural Language Interface Question-Answering Performing Question-Answering with the Fine-Tuned BERT Named Entity Recognition Summary: Getting Hands-on with BERT Quiz: Getting Hands-On with BERT

Exploring BERT Variants

1 Lesson

Different BERT Variants

12 Lessons

BERT Variants—Based on Knowledge Distillation

14 Lessons

Applications of BERT

1 Lesson

Exploring BERTSUM for Text Summarization

8 Lessons

Applying BERT to Other Languages

18 Lessons

Exploring Sentence and Domain-Specific BERT

10 Lessons

Working with VideoBERT, BART, and More

10 Lessons

Conclusion

1 Lesson

COURSE AUTHOR

How You'll Learn

You don’t get better at swimming by watching others. Coding is no different. Practice as you learn with live code environments inside your browser.

Videos are holding you back. Educative‘s interactive, text-based lessons accelerate learning — no setup, downloads, or alt-tabbing required.

Learn faster and smarter with adaptive AI tools embedded in every Educative course.

Built-in assessments let you test your skills. Completion certificates let you show them off.

Recommended Courses

BEFORE STARTING THIS COURSE

AFTER FINISHING THIS COURSE

FOR TEAMS

Interested in this course for your business or team?

Unlock this course (and 1,000+ more) for your entire org with DevPath