Part II
Attention is orderless
What you should noticeShuffle with stamp OFF — every Δ stays ~0. Turn stamp ON, shuffle again — outputs move. That is why transformers need positional encoding.
Transformers mix tokens with attention. Empirically: shuffle tokens with no positional stamp and each token’s output is unchanged (permutation invariance). Add a stamp and order suddenly matters. That is why positional encoding exists.
Loading Godot WASM…
If this stays blank, run godot/tools/export_web.sh 02_attention then rebuild the site.