  Perhaps surprisingly, recent studies have shown probabilistic model
likelihoods have poor specificity for out-of-distribution (OOD) detection and
often assign higher likelihoods to OOD data than in-distribution data. To
ameliorate this issue we propose DoSE, the density of states estimator. Drawing
on the statistical physics notion of ``density of states,'' the DoSE decision
rule avoids direct comparison of model probabilities, and instead utilizes the
``probability of the model probability,'' or indeed the frequency of any
reasonable statistic. The frequency is calculated using nonparametric density
estimators (e.g., KDE and one-class SVM) which measure the typicality of
various model statistics given the training data and from which we can flag
test points with low typicality as anomalous. Unlike many other methods, DoSE
requires neither labeled data nor OOD examples. DoSE is modular and can be
trivially applied to any existing, trained model. We demonstrate DoSE's
state-of-the-art performance against other unsupervised OOD detectors on
previously established ``hard'' benchmarks.
