  We study policy gradient for mean-field control in continuous time in a
reinforcement learning setting. By considering randomised policies with entropy
regularisation, we derive a gradient expectation representation of the value
function, which is amenable to actor-critic type algorithms, where the value
functions and the policies are learnt alternately based on observation samples
of the state and model-free estimation of the population state distribution,
either by offline or online learning. In the linear-quadratic mean-field
framework, we obtain an exact parametrisation of the actor and critic functions
defined on the Wasserstein space. Finally, we illustrate the results of our
algorithms with some numerical experiments on concrete examples.
