  The stochastic mirror descent (SMD) algorithm is a general class of training
algorithms, which includes the celebrated stochastic gradient descent (SGD), as
a special case. It utilizes a mirror potential to influence the implicit bias
of the training algorithm. In this paper we explore the performance of the SMD
iterates on mean-field ensemble models. Our results generalize earlier ones
obtained for SGD on such models. The evolution of the distribution of
parameters is mapped to a continuous time process in the space of probability
distributions. Our main result gives a nonlinear partial differential equation
to which the continuous time process converges in the asymptotic regime of
large networks. The impact of the mirror potential appears through a
multiplicative term that is equal to the inverse of its Hessian and which can
be interpreted as defining a gradient flow over an appropriately defined
Riemannian manifold. We provide numerical simulations which allow us to study
and characterize the effect of the mirror potential on the performance of
networks trained with SMD for some binary classification problems.
