  Model quantization is challenging due to many tedious hyper-parameters such
as precision (bitwidth), dynamic range (minimum and maximum discrete values)
and stepsize (interval between discrete values). Unlike prior arts that
carefully tune these values, we present a fully differentiable approach to
learn all of them, named Differentiable Dynamic Quantization (DDQ), which has
several benefits. (1) DDQ is able to quantize challenging lightweight
architectures like MobileNets, where different layers prefer different
quantization parameters. (2) DDQ is hardware-friendly and can be easily
implemented using low-precision matrix-vector multiplication, making it capable
in many hardware such as ARM. (3) Extensive experiments show that DDQ
outperforms prior arts on many networks and benchmarks, especially when models
are already efficient and compact. e.g., DDQ is the first approach that
achieves lossless 4-bit quantization for MobileNetV2 on ImageNet.
