  The discrepant posterior phenomenon (DPP) is a counter-intuitive phenomenon
that can frequently occur in a Bayesian analysis of multivariate parameters. It
refers to the phenomenon that a parameter estimate based on a posterior is more
extreme than both of those inferred based on either the prior or the likelihood
alone. Inferential claims that exhibit DPP defy the common intuition that the
posterior is a prior-data compromise, and the phenomenon can be surprisingly
ubiquitous in well-behaved Bayesian models. In this paper we revisit this
phenomenon and, using point estimation as an example, derive conditions under
which the DPP occurs in Bayesian models with exponential quadratic likelihoods
and conjugate multivariate Gaussian priors. The family of exponential quadratic
likelihood models includes Gaussian models and those models with local
asymptotic normality property. We provide an intuitive geometric interpretation
of the phenomenon and show that there exists a nontrivial space of marginal
directions such that the DPP occurs. We further relate the phenomenon to the
Simpson's paradox and discover their deep-rooted connection that is associated
with marginalization. We also draw connections with Bayesian computational
algorithms when difficult geometry exists. Our discovery demonstrates that DPP
is more prevalent than previously understood and anticipated. Theoretical
results are complemented by numerical illustrations. Scenarios covered in this
study have implications for parameterization, sensitivity analysis, and prior
choice for Bayesian modeling.
