  Neural network models are resource hungry. It is difficult to deploy such
deep networks on devices with limited resources, like smart wearables,
cellphones, drones, and autonomous vehicles. Low bit quantization such as
binary and ternary quantization is a common approach to alleviate this resource
requirements. Ternary quantization provides a more flexible model and
outperforms binary quantization in terms of accuracy, however doubles the
memory footprint and increases the computational cost. Contrary to these
approaches, mixed quantized models allow a trade-off between accuracy and
memory footprint. In such models, quantization depth is often chosen manually,
or is tuned using a separate optimization routine. The latter requires training
a quantized network multiple times. Here, we propose an adaptive combination of
binary and ternary quantization, namely Smart Quantization (SQ), in which the
quantization depth is modified directly via a regularization function, so that
the model is trained only once. Our experimental results show that the proposed
method adapts quantization depth successfully while keeping the model accuracy
high on MNIST and CIFAR10 benchmarks.
