  Value Iteration Networks (VINs) have emerged as a popular method to
incorporate planning algorithms within deep reinforcement learning, enabling
performance improvements on tasks requiring long-range reasoning and
understanding of environment dynamics. This came with several limitations,
however: the model is not incentivised in any way to perform meaningful
planning computations, the underlying state space is assumed to be discrete,
and the Markov decision process (MDP) is assumed fixed and known. We propose
eXecuted Latent Value Iteration Networks (XLVINs), which combine recent
developments across contrastive self-supervised learning, graph representation
learning and neural algorithmic reasoning to alleviate all of the above
limitations, successfully deploying VIN-style models on generic environments.
XLVINs match the performance of VIN-like models when the underlying MDP is
discrete, fixed and known, and provides significant improvements to model-free
baselines across three general MDP setups.
