  Deep neural networks have achieved state-of-the art performance on various
computer vision tasks. However, their deployment on resource-constrained
devices has been hindered due to their high computational and storage
complexity. While various complexity reduction techniques, such as lightweight
network architecture design and parameter quantization, have been successful in
reducing the cost of implementing these networks, these methods have often been
considered orthogonal. In reality, existing quantization techniques fail to
replicate their success on lightweight architectures such as MobileNet. To this
end, we present a novel fully differentiable non-uniform quantizer that can be
seamlessly mapped onto efficient ternary-based dot product engines. We conduct
comprehensive experiments on CIFAR-10, ImageNet, and Visual Wake Words
datasets. The proposed quantizer (DBQ) successfully tackles the daunting task
of aggressively quantizing lightweight networks such as MobileNetV1,
MobileNetV2, and ShuffleNetV2. DBQ achieves state-of-the art results with
minimal training overhead and provides the best (pareto-optimal)
accuracy-complexity trade-off.
