  We systematically review the Variational Optimization, Variational Inference
and Stochastic Search perspectives on sampling-based dynamic optimization and
discuss their connections to state-of-the-art optimizers and Stochastic Optimal
Control (SOC) theory. A general convergence and sample complexity analysis on
the three perspectives is provided through the unifying Stochastic Search
perspective. We then extend these frameworks to their distributed versions for
multi-agent control by combining them with consensus Alternating Direction
Method of Multipliers (ADMM) to decouple the full problem into local
neighborhood-level ones that can be solved in parallel. Model Predictive
Control (MPC) algorithms are then developed based on these frameworks, leading
to fully decentralized sampling-based dynamic optimizers. The capabilities of
the proposed algorithms framework are demonstrated on multiple complex
multi-agent tasks for vehicle and quadcopter systems in simulation. The results
compare different distributed sampling-based optimizers and their centralized
counterparts using unimodal Gaussian, mixture of Gaussians, and stein
variational policies. The scalability of the proposed distributed algorithms is
demonstrated on a 196-vehicle scenario where a direct application of
centralized sampling-based methods is shown to be prohibitive.
