  The celebrated Bernstein von-Mises theorem ensures that credible regions from
Bayesian posterior are well-calibrated when the model is correctly-specified,
in the frequentist sense that their coverage probabilities tend to the nominal
values as data accrue. However, this conventional Bayesian framework is known
to lack robustness when the model is misspecified or only partly specified,
such as in quantile regression, risk minimization based supervised/unsupervised
learning and robust estimation. To overcome this difficulty, we propose a new
Bayesian inferential approach that substitutes the (misspecified or partly
specified) likelihoods with proper exponentially tilted empirical likelihoods
plus a regularization term. Our surrogate empirical likelihood is carefully
constructed by using the first order optimality condition of the empirical risk
minimization as the moment condition. We show that the Bayesian posterior
obtained by combining this surrogate empirical likelihood and the prior is
asymptotically close to a normal distribution centering at the empirical risk
minimizer with covariance matrix taking an appropriate sandwiched form.
Consequently, the resulting Bayesian credible regions are automatically
calibrated to deliver valid uncertainty quantification. Computationally, the
proposed method can be easily implemented by Markov Chain Monte Carlo sampling
algorithms. Our numerical results show that the proposed method tends to be
more accurate than existing state-of-the-art competitors.
