  Many modern applications use computer vision to detect and count objects in
massive image collections. However, when the detection task is very difficult
or in the presence of domain shifts, the counts may be inaccurate even with
significant investments in training data and model development. We propose
DISCount -- a detector-based importance sampling framework for counting in
large image collections that integrates an imperfect detector with
human-in-the-loop screening to produce unbiased estimates of counts. We propose
techniques for solving counting problems over multiple spatial or temporal
regions using a small number of screened samples and estimate confidence
intervals. This enables end-users to stop screening when estimates are
sufficiently accurate, which is often the goal in a scientific study. On the
technical side we develop variance reduction techniques based on control
variates and prove the (conditional) unbiasedness of the estimators. DISCount
leads to a 9-12x reduction in the labeling costs over naive screening for tasks
we consider, such as counting birds in radar imagery or estimating damaged
buildings in satellite imagery, and also surpasses alternative covariate-based
screening approaches in efficiency.
