Transformers Architecture Quiz

What you'll be asked

1. What is the primary purpose of the 'Self-Attention' mechanism in a Transformer model?
2. In the original Transformer architecture, why are 'Positional Encodings' added to the input embeddings?
3. Which component of the Transformer architecture is specifically responsible for mapping the output of the attention layers to a higher-dimensional space and then back to the original dimension?
4. What is the role of 'Multi-Head Attention' in a Transformer?
5. In a Transformer decoder, why is 'Masked Multi-Head Attention' used during training?

Answers and explanations come after you submit.