Transformer Visualization with BertViz
Learn about how layers of a transformer model are visualized with BertViz.
Jesse Vig’s article, "
BertViz can visualize attention head activity and interpret a transformer model’s behavior.
BertViz was first designed to visualize BERT and GPT-3 models. In this lesson, we will visualize the activity of a BERT model.
Let’s now run BertViz.
Running BertViz
It only takes five steps to visualize transformer attention heads and interact with them.
Step 1: Importing BertViz and other modules
The notebook BertViz.ipynb
imports bertViz
, Hugging Face transformers, and the other basic requirements to implement the program:
Get hands-on with 1200+ tech skills courses.