  Enabling low precision implementations of deep learning models, without
considerable performance degradation, is necessary in resource and latency
constrained settings. Moreover, exploiting the differences in sensitivity to
quantization across layers can allow mixed precision implementations to achieve
a considerably better computation performance trade-off. However,
backpropagating through the quantization operation requires introducing
gradient approximations, and choosing which layers to quantize is challenging
for modern architectures due to the large search space. In this work, we
present a constrained learning approach to quantization aware training. We
formulate low precision supervised learning as a constrained optimization
problem, and show that despite its non-convexity, the resulting problem is
strongly dual and does away with gradient estimations. Furthermore, we show
that dual variables indicate the sensitivity of the objective with respect to
constraint perturbations. We demonstrate that the proposed approach exhibits
competitive performance in image classification tasks, and leverage the
sensitivity result to apply layer selective quantization based on the value of
dual variables, leading to considerable performance improvements.
