Same building blocks as before — a different trick
Same token vectors, same frequency spectrum as the additive page — but instead of adding a position signal, RoPE rotates the vector itself. Drag any outlined handle; the sliders are a backup.
← → nudge the last slider · space play the last sweep · Esc leave tour
Two tokens, always 5 slots apart. Slide them through the sequence — the gap never changes, yet the similarity does.
Click the curve to jump to a shift, or hover it to read a value. Same-gap pairs never matched on the additive page — this is why.
Everything RoPE does comes from one object: a 2×2 rotation matrix. Drag the black arrowhead and watch what it preserves.
Its two columns are where (1,0) and (0,1) land — that is the whole matrix.
Split the vector into d/2 pairs, each with its own frequency — the same spectrum as the additive page. Now each pair is rotated instead of added to.
Click a clock to zoom in, then drag its hand. Scroll on a clock to zoom.
Only d/2 tiny 2×2 blocks are non-zero; the rest is 0. The active block is outlined.
Blank cells are 0 — d/2 little rotations side by side.
The flat embedding first splits into coloured pairs, then each pair rotates. Click a pair to inspect it.
Each bracket of two bars is one pair — same colour as its clock.
Query at position m, key at position n, in the same pair's plane. Drag either arrowhead and watch the angle and the dot product.
Transpose reverses a rotation, and rotations compose by adding angles — so the score only sees n − m.
Each cell is a score between m (row) and n (column), summed over all d/2 pairs. Click a cell to pin its diagonal, or press play.