Part I
The JEPA pivot
What you should noticeJEPA predicts future latents. Anti-collapse (e.g. SIGReg) stops the encoder from mapping everything to one point.
JEPA keeps the encoder idea and the “no pixel recon” idea, then adds the missing piece: predict the future latent given the current latent and an action.
Without an anti-collapse term the encoder can cheat by mapping everything to one point (prediction becomes trivial). LeWM uses SIGReg to keep latents informative and roughly isotropic. Next: the transformer mechanisms that build those latents.