  Humans and animals show remarkable flexibility in adjusting their behaviour
when their goals, or rewards in the environment change. While such flexibility
is a hallmark of intelligent behaviour, these multi-task scenarios remain an
important challenge for machine learning algorithms and neurobiological models
alike. We investigated two approaches that could enable this flexibility:
factorized representations, which abstract away general aspects of a task from
those prone to change, and nonparametric, memory-based approaches, which can
provide a principled way of using similarity to past experiences to guide
current behaviour. In particular, we combine the successor representation (SR)
that factors the value of actions into expected outcomes and corresponding
rewards with evaluating task similarity through clustering the space of reward
functions. The proposed algorithm inverts a generative model over tasks, and
dynamically samples from a flexible number of distinct SR maps while
accumulating evidence about the current task context through amortized
inference. It improves SR's transfer capabilities and outperforms competing
algorithms and baselines in settings with both known and unsignalled rewards
changes. Further, as a neurobiological model of spatial coding in the
hippocampus, it explains important signatures of this representation, such as
the "flickering" behaviour of hippocampal maps, and trajectory-dependent place
cells (so-called splitter cells) and their dynamics. We thus provide a novel
algorithmic approach for multi-task learning, as well as a common normative
framework that links together these different characteristics of the brain's
spatial representation.
