  The Bayesian method is noted to produce spuriously high posterior
probabilities for phylogenetic trees in analysis of large datasets, but the
precise reasons for this over-confidence are unknown. In general, the
performance of Bayesian selection of misspecified models is poorly understood,
even though this is of great scientific interest since models are never true in
real data analysis. Here we characterize the asymptotic behavior of Bayesian
model selection and show that when the competing models are equally wrong,
Bayesian model selection exhibits surprising and polarized behaviors in large
datasets, supporting one model with full force while rejecting the others. If
one model is slightly less wrong than the other, the less wrong model will
eventually win when the amount of data increases, but the method may become
overconfident before it becomes reliable. We suggest that this extreme behavior
may be a major factor for the spuriously high posterior probabilities for
evolutionary trees. The philosophical implications of our results to the
application of Bayesian model selection to evaluate opposing scientific
hypotheses are yet to be explored, as are the behaviors of non-Bayesian methods
in similar situations.
