  Assessing the performance of a learned model is a crucial part of machine
learning. However, in some domains only positive and unlabeled examples are
available, which prohibits the use of most standard evaluation metrics. We
propose an approach to estimate any metric based on contingency tables,
including ROC and PR curves, using only positive and unlabeled data. Estimating
these performance metrics is essentially reduced to estimating the fraction of
(latent) positives in the unlabeled set, assuming known positives are a random
sample of all positives. We provide theoretical bounds on the quality of our
estimates, illustrate the importance of estimating the fraction of positives in
the unlabeled set and demonstrate empirically that we are able to reliably
estimate ROC and PR curves on real data.
