  Binary Neural Networks (BNNs) have emerged as a promising solution for
reducing the memory footprint and compute costs of deep neural networks, but
they suffer from quality degradation due to the lack of freedom as activations
and weights are constrained to the binary values. To compensate for the
accuracy drop, we propose a novel BNN design called Binary Neural Network with
INSTAnce-aware threshold (INSTA-BNN), which controls the quantization threshold
dynamically in an input-dependent or instance-aware manner. According to our
observation, higher-order statistics can be a representative metric to estimate
the characteristics of the input distribution. INSTA-BNN is designed to adjust
the threshold dynamically considering various information, including
higher-order statistics, but it is also optimized judiciously to realize
minimal overhead on a real device. Our extensive study shows that INSTA-BNN
outperforms the baseline by 3.0% and 2.8% on the ImageNet classification task
with comparable computing cost, achieving 68.5% and 72.2% top-1 accuracy on
ResNet-18 and MobileNetV1 based models, respectively.
