  We introduce a new online algorithm for expected log-likelihood maximization
in situations where the objective function is multi-modal and/or has saddle
points, that we term G-PFSO. The key element underpinning G-PFSO is a
probability distribution which (a) is shown to concentrate on the target
parameter value as the sample size increases and (b) can be efficiently
estimated by means of a standard particle filter algorithm. This distribution
depends on a learning rate, where the faster the learning rate the quicker it
concentrates on the desired element of the search space, but the less likely
G-PFSO is to escape from a local optimum of the objective function. In order to
achieve a fast convergence rate with a slow learning rate, G-PFSO exploits the
acceleration property of averaging, well-known in the stochastic gradient
literature. Considering several challenging estimation problems, the numerical
experiments show that, with high probability, G-PFSO successfully finds the
highest mode of the objective function and converges to its global maximizer at
the optimal rate. While the focus of this work is expected log-likelihood
maximization, the proposed methodology and its theory apply more generally for
optimizing a function defined through an expectation.
