  The normalized maximum likelihood (NML) is a recent penalized likelihood that
has properties that justify defining the amount of discrimination information
(DI) in the data supporting an alternative hypothesis over a null hypothesis as
the logarithm of an NML ratio, namely, the alternative hypothesis NML divided
by the null hypothesis NML. The resulting DI, like the Bayes factor but unlike
the p-value, measures the strength of evidence for an alternative hypothesis
over a null hypothesis such that the probability of misleading evidence
vanishes asymptotically under weak regularity conditions and such that evidence
can support a simple null hypothesis. Unlike the Bayes factor, the DI does not
require a prior distribution and is minimax optimal in a sense that does not
involve averaging over outcomes that did not occur. Replacing a (possibly
pseudo-) likelihood function with its weighted counterpart extends the scope of
the DI to models for which the unweighted NML is undefined. The likelihood
weights leverage side information, either in data associated with comparisons
other than the comparison at hand or in the parameter value of a simple null
hypothesis. Two case studies, one involving multiple populations and the other
involving multiple biological features, indicate that the DI is robust to the
type of side information used when that information is assigned the weight of a
single observation. Such robustness suggests that very little adjustment for
multiple comparisons is warranted if the sample size is at least moderate.
