  Many common methods for learning a world model for pixel-based environments
use generative architectures trained with pixel-level reconstruction
objectives. Recently proposed Joint Embedding Predictive Architectures (JEPA)
offer a reconstruction-free alternative. In this work, we analyze performance
of JEPA trained with VICReg and SimCLR objectives in the fully offline setting
without access to rewards, and compare the results to the performance of the
generative architecture. We test the methods in a simple environment with a
moving dot with various background distractors, and probe learned
representations for the dot's location. We find that JEPA methods perform on
par or better than reconstruction when distractor noise changes every time
step, but fail when the noise is fixed. Furthermore, we provide a theoretical
explanation for the poor performance of JEPA-based methods with fixed noise,
highlighting an important limitation.
