  This paper presents approaches to mean-field control, motivated by
distributed control of multi-agent systems. Control solutions are based on a
convex optimization problem, whose domain is a convex set of probability mass
functions (pmfs). The main contributions follow: 1. Kullback-Leibler-Quadratic
(KLQ) optimal control is a special case, in which the objective function is
composed of a control cost in the form of Kullback-Leibler divergence between a
candidate pmf and the nominal, plus a quadratic cost on the sequence of
marginals. Theory in this paper extends prior work on deterministic control
systems, establishing that the optimal solution is an exponential tilting of
the nominal pmf. Transform techniques are introduced to reduce complexity of
the KLQ solution, motivated by the need to consider time horizons that are much
longer than the inter-sampling times required for reliable control. 2.
Infinite-horizon KLQ leads to a state feedback control solution with attractive
properties. It can be expressed as either state feedback, in which the state is
the sequence of marginal pmfs, or an open loop solution is obtained that is
more easily computed. 3. Numerical experiments are surveyed in an application
of distributed control of residential loads to provide grid services, similar
to utility-scale battery storage. The results show that KLQ optimal control
enables the aggregate power consumption of a collection of flexible loads to
track a time-varying reference signal, while simultaneously ensuring each
individual load satisfies its own quality of service constraints. Keywords:
Mean field games, distributed control, Markov decision processes, Demand
Dispatch.
