The Transformer, piece by piece

Explore exposes every technical detail. Classroom gives a concise presenter-led walkthrough. Story builds the intuition as one visual narrative.

Running example: “I love deep learning” → 我 · 爱 · 深度学习

Encoder Decoder Cross-Attention / memory Q K V
1

Encoder

Self-Attention + FFN, bidirectional attention, contextual output H.

Tip: click a token (bottom of the stack) or a row of the matrix to change the query token.

2

Decoder

Masked Self-Attention + Cross-Attention + FFN, generating one token at a time.

3

Encoder + Decoder = Transformer

Two stacks, one bridge: the Encoder output H is wired into every Decoder layer.

4

Information flow: Cross-Attention

How the Encoder output is injected into the Decoder — Q from the Decoder, K/V from the Encoder.

Tip: click a Decoder token (right) or a row of the matrix to change which token is doing the querying.

5

Attention Masks

Padding mask vs. look-ahead mask — where they apply and how their shapes differ.

Where is the mask applied?
6

Why these attention directions?

Bidirectional in the Encoder, causal in the Decoder — and what breaks if you swap them.

Encoder attention
Decoder self-attention
01 / 08
ACT I · WHY TWO ROLES?One sequence is read; one is written
← → navigate · Space replay