Understanding Multilingual BERT
Explore the fundamentals of Multilingual BERT and how it can represent text in more than 100 languages using a shared vocabulary and training approach. Learn to apply the pre-trained M-BERT model with libraries like Hugging Face to generate multilingual sentence representations for various NLP tasks.
We'll cover the following...
BERT provides representation for only English text. Let's suppose we have an input text in a different language, say, French. Now, how can we use BERT to obtain a representation of the French text? Here is where we use Multilingual BERT (M-BERT).
M-BERT
Multilingual BERT is used to obtain representations of text in different languages and not just English. We learned that the BERT model is trained with masked language modeling (MLM) and next sentence prediction (NSP) tasks using the English Wikipedia text and the Toronto BookCorpus. Similar to BERT, M-BERT is also trained with MLM and NSP tasks, but instead of using the Wikipedia text of only the English language, M-BERT is trained using the Wikipedia text of 104 different languages. ...