  We revisit the role of instrumental value as a driver of adaptive behavior.
In active inference, instrumental or extrinsic value is quantified by the
information-theoretic surprisal of a set of observations measuring the extent
to which those observations conform to prior beliefs or preferences. That is,
an agent is expected to seek the type of evidence that is consistent with its
own model of the world. For reinforcement learning tasks, the distribution of
preferences replaces the notion of reward. We explore a scenario in which the
agent learns this distribution in a self-supervised manner. In particular, we
highlight the distinction between observations induced by the environment and
those pertaining more directly to the continuity of an agent in time. We
evaluate our methodology in a dynamic environment with discrete time and
actions. First with a surprisal minimizing model-free agent (in the RL sense)
and then expanding to the model-based case to minimize the expected free
energy.
