  Analysis of credibility is a reverse-Bayes technique that has been proposed
by Matthews (2001) to overcome some of the shortcomings of significance tests.
A significant result is deemed credible if current knowledge about the effect
size is in conflict with any sceptical prior that would make the effect
non-significant. In this paper I formalize the approach and propose to use
Bayesian predictive tail probabilities to quantify the evidence for
credibility. This gives rise to a p-value for extrinsic credibility, taking
into account both the internal and the external evidence for an effect. The
assessment of intrinsic credibility leads to a new threshold for ordinary
significance that is remarkably close to the recently proposed 0.005 level.
Finally, a p-value for intrinsic credibility is proposed that is a simple
function of the ordinary p-value for significance and has a direct frequentist
interpretation in terms of the replication probability that a future study
under identical conditions will give an estimated effect in the same direction
as the first study.
