  Let $(Y_t)_{t\geq 1}$ be a sequence of i.i.d.\ observations and
$\{f_\theta,\theta\in \mathbb{R}^d\}$ be a parametric model. We introduce a new
online algorithm for computing a sequence $(\hat{\theta}_t)_{t\geq 1}$ which is
shown to converge almost surely to $\text{argmax}_{\theta\in
\mathbb{R}^d}\mathbb{E}[\log f_\theta(Y_1)]$ at rate $ \mathcal{O}(\log
(t)^{(1+\varepsilon)/2}t^{-1/2})$, with $\varepsilon>0$ a user specified
parameter. This convergence result is obtained under standard conditions on the
statistical model and, most notably, we allow the mapping $\theta\mapsto
\mathbb{E}[\log f_\theta(Y_1)]$ to be multi-modal. However, the computational
cost to process each observation grows exponentially with the dimension of
$\theta$, which makes the proposed approach applicable to low or moderate
dimensional problems only. We also derive a version of the estimator
$\hat{\theta}_t$ which is well suited to Student-t linear regression models.
The corresponding estimator of the regression coefficients is robust to the
presence of outliers, as shown by experiments on simulated and real data, and
thus, as a by-product of this work, we obtain a new online and adaptive robust
estimation method for linear regression models.
