Part II
Attention fans + multi-head
What you should noticeToggle heads — each has its own Q/K, so the fan pattern changes over the same tokens.
Softmax weights become a visible fan. Multiple heads run separate Q/K projections in parallel — here random, to show the mechanism. Trained heads may specialize; many are also redundant. LeWM uses few heads in the encoder and many in the predictor.
Loading Godot WASM…
If this stays blank, run godot/tools/export_web.sh 03_attention_viz then rebuild the site.