  In this paper we provide a rigorous convergence analysis for the renowned
particle swarm optimization method by using tools from stochastic calculus and
the analysis of partial differential equations. Based on a time-continuous
formulation of the particle dynamics as a system of stochastic differential
equations, we establish convergence to a global minimizer of a possibly
nonconvex and nonsmooth objective function in two steps. First, we prove
consensus formation of an associated mean-field dynamics by analyzing the
time-evolution of the variance of the particle distribution. We then show that
this consensus is close to a global minimizer by employing the asymptotic
Laplace principle and a tractability condition on the energy landscape of the
objective function. These results allow for the usage of memory mechanisms, and
hold for a rich class of objectives provided certain conditions of
well-preparation of the hyperparameters and the initial datum. In a second
step, at least for the case without memory effects, we provide a quantitative
result about the mean-field approximation of particle swarm optimization, which
specifies the convergence of the interacting particle system to the associated
mean-field limit. Combining these two results allows for global convergence
guarantees of the numerical particle swarm optimization method with provable
polynomial complexity. To demonstrate the applicability of the method we
propose an efficient and parallelizable implementation, which is tested in
particular on a competitive and well-understood high-dimensional benchmark
problem in machine learning.
