  Building reliable AI decision support systems requires a robust set of data
on which to train models; both with respect to quantity and diversity.
Obtaining such datasets can be difficult in resource limited settings, or for
applications in early stages of deployment. Sample rejection is one way to work
around this challenge, however much of the existing work in this area is
ill-suited for such scenarios. This paper substantiates that position and
proposes a simple solution as a proof of concept baseline.
