  In this paper, we propose a new stochastic optimization algorithm for
Bayesian inference based on multilevel Monte Carlo (MLMC) methods. In Bayesian
statistics, biased estimators of the model evidence have been often used as
stochastic objectives because the existing debiasing techniques are
computationally costly to apply. To overcome this issue, we apply an MLMC
sampling technique to construct low-variance unbiased estimators both for the
model evidence and its gradient. In the theoretical analysis, we show that the
computational cost required for our proposed MLMC estimator to estimate the
model evidence or its gradient with a given accuracy is an order of magnitude
smaller than those of the previously known estimators. Our numerical
experiments confirm considerable computational savings compared to the
conventional estimators. Combining our MLMC estimator with gradient-based
stochastic optimization results in a new scalable, efficient, debiased
inference algorithm for Bayesian statistical models.
