M-BERT Generalization

Understand how M-BERT handles cross-lingual generalization in POS tagging by exploring its performance on languages with varying scripts and syntactic orders. Learn how typological and structural similarities between languages influence M-BERT's zero-shot transfer capabilities.

We'll cover the following...

Generalization across scripts
Generalization across typological features
- Accuracy of the POS tagging task
Effect of language similarity

Generalization across scripts

Let's investigate whether the M-BERT model can generalize across languages that are written in different scripts. Let's conduct a small experiment to understand this. Say we are performing a POS (part of speech) tagging task. First, we fine-tune M-BERT for the POS Part-of-speechtagging task in the Urdu language. Then, we evaluate the fine-tuned M-BERT model in a different language, say Hindi. Note that Urdu follows Arabic script, while Hindi follows Devanagari script. A simple example of Urdu and Hindi text is given here:

1.Before We Start

2.Starting Off with BERT

3.A Primer on Transformers

Project

4.Understanding the BERT Model

5.Getting Hands-On with BERT

6.Exploring BERT Variants

7.Different BERT Variants

8.BERT Variants—Based on Knowledge Distillation

9.Applications of BERT

10.Exploring BERTSUM for Text Summarization

11.Applying BERT to Other Languages

12.Exploring Sentence and Domain-Specific BERT

13.Working with VideoBERT, BART, and More

14.Conclusion

Project

M-BERT Generalization

Generalization across scripts