Every token becomes a reader
A = softmax
row
( Q K
⊤
/ √d
k
) Z = A V
Next reader
Play
Reset
① Stages
② Attention matrix
reader ↓ source →
③ One reader
weights → mixture
Reader
View