Search⌘ K
AI Features
Home
Courses
Getting Started with Google BERT

Chinese BERT

Explore the Chinese BERT model, understanding its architecture based on BERT-base and its pre-training with whole word masking (WWM). Learn how WWM enhances training by masking entire words instead of subwords, and discover how Chinese text segmentation aids in processing. This lesson equips you to utilize the pre-trained Chinese BERT model with transformers libraries for effective Chinese NLP applications.

We'll cover the following...

Along with M-BERT, Google Research has also open sourced the Chinese BERT model. The configuration of the Chinese BERT model is the same as the vanilla BERT-base model. It consists of 12 encoder layers, 12 attention heads, and 768 hidden units with 110 million parameters. The pre-trained Chinese BERT model can be downloaded from GitHub.

We can use the pre-trained Chinese BERT model with the transformers library, as shown here: 

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("bert-base-chinese")
model = AutoModel.from_pretrained("bert-base-chinese")

Now, let's look into another ...