  Posterior predictive p-values are a common approach to Bayesian
model-checking. This article analyses their frequency behaviour, that is, their
distribution when the parameters and the data are drawn from the prior and the
model respectively. We show that the family of possible distributions is
exactly described as the distributions that are less variable than uniform on
[0,1], in the convex order. In general, p-values with such a property are not
conservative, and we illustrate how the theoretical worst-case error rate for
false rejection can occur in practice. We describe how to correct the p-values
to recover conservatism in several common scenarios, for example, when
interpreting a single p-value or when combining multiple p-values into an
overall score of significance. We also handle the case where the p-value is
estimated from posterior samples obtained from techniques such as Markov Chain
or Sequential Monte Carlo. Our results place posterior predictive p-values in a
much clearer theoretical framework, allowing them to be used with more
assurance.
