  In this work we consider partially observable environments with sparse
rewards. We present a self-supervised representation learning method for
image-based observations, which arranges embeddings respecting temporal
distance of observations. This representation is empirically robust to
stochasticity and suitable for novelty detection from the error of a predictive
forward model. We consider episodic and life-long uncertainties to guide the
exploration. We propose to estimate the missing information about the
environment with the world model, which operates in the learned latent space.
As a motivation of the method, we analyse the exploration problem in a tabular
Partially Observable Labyrinth. We demonstrate the method on image-based hard
exploration environments from the Atari benchmark and report significant
improvement with respect to prior work. The source code of the method and all
the experiments is available at https://github.com/htdt/lwm.
