  Recent advances in deep learning have made the use of large, deep neural
networks with tens of millions of parameters. The sheer size of these networks
imposes a challenging computational burden during inference. Existing work
focuses primarily on accelerating each forward pass of a neural network.
Inspired by the group testing strategy for efficient disease testing, we
propose neural group testing, which accelerates by testing a group of samples
in one forward pass. Groups of samples that test negative are ruled out. If a
group tests positive, samples in that group are then retested adaptively. A key
challenge of neural group testing is to modify a deep neural network so that it
could test multiple samples in one forward pass. We propose three designs to
achieve this without introducing any new parameters and evaluate their
performances. We applied neural group testing in an image moderation task to
detect rare but inappropriate images. We found that neural group testing can
group up to 16 images in one forward pass and reduce the overall computation
cost by over 73% while improving detection performance.
