Different heads read differently
MHA = Concat(head
1
…head
h
) W
O
Add a head
Play
Reset
① Heads
one reader, every head
② Weights
the selected head, every reader
③ Combine
concat → W
O
Heads
Reader