  This paper studies the semi-supervised novelty detection problem where a set
of "typical" measurements is available to the researcher. Motivated by recent
advances in multiple testing and conformal inference, we propose AdaDetect, a
flexible method that is able to wrap around any probabilistic classification
algorithm and control the false discovery rate (FDR) on detected novelties in
finite samples without any distributional assumption other than
exchangeability. In contrast to classical FDR-controlling procedures that are
often committed to a pre-specified p-value function, AdaDetect learns the
transformation in a data-adaptive manner to focus the power on the directions
that distinguish between inliers and outliers. Inspired by the multiple testing
literature, we further propose variants of AdaDetect that are adaptive to the
proportion of nulls while maintaining the finite-sample FDR control. The
methods are illustrated on synthetic datasets and real-world datasets,
including an application in astrophysics.
