  We consider a class of stochastic control problems where the state process is
a probability measure-valued process satisfying an additional martingale
condition on its dynamics, called measure-valued martingales (MVMs). We
establish the `classical' results of stochastic control for these problems:
specifically, we prove that the value function for the problem can be
characterised as the unique solution to the Hamilton-Jacobi-Bellman equation in
the sense of viscosity solutions. In order to prove this result, we exploit
structural properties of the MVM processes. Our results also include an
appropriate version of It\^o's lemma for controlled MVMs. We also show how
problems of this type arise in a number of applications, including
model-independent derivatives pricing, the optimal Skorokhod embedding problem,
and two player games with asymmetric information.
