  With the advent of high-performance computing, Bayesian methods are
increasingly popular tools for the quantification of uncertainty throughout
science and industry. Since these methods impact the making of sometimes
critical decisions in increasingly complicated contexts, the sensitivity of
their posterior conclusions with respect to the underlying models and prior
beliefs is a pressing question for which there currently exist positive and
negative results. We report new results suggesting that, although Bayesian
methods are robust when the number of possible outcomes is finite or when only
a finite number of marginals of the data-generating distribution are unknown,
they could be generically brittle when applied to continuous systems (and their
discretizations) with finite information on the data-generating distribution.
If closeness is defined in terms of the total variation metric or the matching
of a finite system of generalized moments, then (1) two practitioners who use
arbitrarily close models and observe the same (possibly arbitrarily large
amount of) data may reach opposite conclusions; and (2) any given prior and
model can be slightly perturbed to achieve any desired posterior conclusions.
The mechanism causing brittlenss/robustness suggests that learning and
robustness are antagonistic requirements and raises the question of a missing
stability condition for using Bayesian Inference in a continuous world under
finite information.
