  Active inference offers a first principle account of sentient behaviour, from
which special and important cases can be derived, e.g., reinforcement learning,
active learning, Bayes optimal inference, Bayes optimal design, etc. Active
inference resolves the exploitation-exploration dilemma in relation to prior
preferences, by placing information gain on the same footing as reward or
value. In brief, active inference replaces value functions with functionals of
(Bayesian) beliefs, in the form of an expected (variational) free energy. In
this paper, we consider a sophisticated kind of active inference, using a
recursive form of expected free energy. Sophistication describes the degree to
which an agent has beliefs about beliefs. We consider agents with beliefs about
the counterfactual consequences of action for states of affairs and beliefs
about those latent states. In other words, we move from simply considering
beliefs about 'what would happen if I did that' to 'what would I believe about
what would happen if I did that'. The recursive form of the free energy
functional effectively implements a deep tree search over actions and outcomes
in the future. Crucially, this search is over sequences of belief states, as
opposed to states per se. We illustrate the competence of this scheme, using
numerical simulations of deep decision problems.
