  Quantifying the concordance between different cosmological experiments is
important for testing the validity of theoretical models and systematics in the
observations. In earlier work, we thus proposed the Surprise, a concordance
measure derived from the relative entropy between posterior distributions. We
revisit the properties of the Surprise and describe how it provides a general,
versatile, and robust measure for the agreement between datasets. We also
compare it to other measures of concordance that have been proposed for
cosmology. As an application, we extend our earlier analysis and use the
Surprise to quantify the agreement between WMAP 9, Planck 13 and Planck 15
constraints on the $\Lambda$CDM model. Using a principle component analysis in
parameter space, we find that the large Surprise between WMAP 9 and Planck 13
(S = 17.6 bits, implying a deviation from consistency at 99.8% confidence) is
due to a shift along a direction that is dominated by the amplitude of the
power spectrum. The Planck 15 constraints deviate from the Planck 13 results (S
= 56.3 bits), primarily due to a shift in the same direction. The Surprise
between WMAP and Planck consequently disappears when moving to Planck 15 (S =
-5.1 bits). This means that, unlike Planck 13, Planck 15 is not in tension with
WMAP 9. These results illustrate the advantages of the relative entropy and the
Surprise for quantifying the disagreement between cosmological experiments and
more generally as an information metric for cosmology.
