Quiz: Pretraining a RoBERTa Model from Scratch

Test yourself on the concepts you learned this chapter.

1

We have a custom-made RoBERTa model with layers of dimensions:

  • 1536 x 1536 (repeating 10 times)
  • 1536 x 1 (repeating 10 times)
  • 1536

What is the total number of parameters of this model?

A)

24,608,000

B)

23,609,856

C)

25,612,748

D)

22,987,126

Question 1 of 50 attempted

Get hands-on with 1200+ tech skills courses.