  Bayesian evidence ratios give a very attractive way of comparing models, and
being able to quote the odds on a particular model seems a very clear
motivation for making a choice. Jeffreys' scale of evidence is often used in
the interpretation of evidence ratios. A natural question is, how often will
you get it right when you choose on the basis of some threshold value of the
evidence ratio? The evidence ratio will be different in different realizations
of the data, and its utility can be examined in a Neyman-Pearson like way to
see what the trade-offs are between statistical power (the chance of ``getting
it right'') versus the false alarm rate, picking the alternative hypothesis
when the null is actually true. I will show some simple examples which show
that there can be a surprisingly large range for an evidence ratio under
different realizations of the data. It seems best not to simply rely on
Jeffrey's scale when decisions have to be taken, but also to examine the
probability of taking the ``wrong'' decision if some evidence ratio is taken to
be decisive. Interestingly, Turing knew this and applied it during WWII,
although (like much else) he did not publish it.
