  Testing the equality of two proportions is a common procedure in science,
especially in medicine and public health. In these domains it is crucial to be
able to quantify evidence for the absence of a treatment effect. Bayesian
hypothesis testing by means of the Bayes factor provides one avenue to do so,
requiring the specification of prior distributions for parameters. The most
popular analysis approach views the comparison of proportions from a
contingency table perspective, assigning prior distributions directly to the
two proportions. Another, less popular approach views the problem from a
logistic regression perspective, assigning prior distributions to
logit-transformed parameters. Reanalyzing 39 null results from the New England
Journal of Medicine with both approaches, we find that they can lead to
markedly different conclusions, especially when the observed proportions are at
the extremes (i.e., very low or very high). We explain these stark differences
and provide recommendations for researchers interested in testing the equality
of two proportions and users of Bayes factors more generally. The test that
assigns prior distributions to logit-transformed parameters creates prior
dependence between the two proportions and yields weaker evidence when the
observations are at the extremes. When comparing two proportions, we argue that
this test should become the new default.
