  We examine mean field control problems on a finite state space, in continuous
time and over a finite time horizon. We characterize the value function of the
mean field control problem as the unique viscosity solution of a
Hamilton-Jacobi-Bellman equation in the simplex. In absence of any convexity
assumption, we exploit this characterization to prove convergence, as $N$
grows, of the value functions of the centralized $N$-agent optimal control
problem to the limit mean field control problem value function, with a
convergence rate of order $1/\sqrt{N}$. Then, assuming convexity, we show that
the limit value function is smooth and establish propagation of chaos, i.e.
convergence of the $N$-agent optimal trajectory to the unique limiting optimal
trajectory, with an explicit rate.
