  We present a simple theoretical framework, and corresponding practical
procedures, for comparing probabilistic models on real data in a traditional
machine learning setting. This framework is based on the theory of proper
scoring rules, but requires only basic algebra and probability theory to
understand and verify. The theoretical concepts presented are well-studied,
primarily in the statistics literature. The goal of this paper is to advocate
their wider adoption for performance evaluation in empirical machine learning.
