  Many measurements in the physical sciences can be cast as counting
experiments, where the number of occurrences of a physical phenomenon informs
the prevalence of the phenomenon's source. Often, detection of the physical
phenomenon (termed signal) is difficult to distinguish from naturally occurring
phenomena (termed background). In this case, the discrimination of signal
events from background can be performed using classifiers, and they may range
from simple, threshold-based classifiers to sophisticated neural networks.
These classifiers are often trained and validated to obtain optimal accuracy,
however we show that the optimal accuracy classifier does not generally
coincide with a classifier that provides the lowest detection limit, nor the
lowest quantification uncertainty. We present a derivation of the detection
limit and quantification uncertainty in the classifier-based counting
experiment case. We also present a novel abstention mechanism to minimize the
detection limit or quantification uncertainty \emph{a posteriori}. We
illustrate the method on two data sets from the physical sciences,
discriminating Ar-37 and Ar-39 radioactive decay from non-radioactive events in
a gas proportional counter, and discriminating neutrons from photons in an
inorganic scintillator and report results therefrom.
