  To bridge the ever increasing gap between deep neural networks' complexity
and hardware capability, network quantization has attracted more and more
research attention. The latest trend of mixed precision quantization takes
advantage of hardware's multiple bit-width arithmetic operations to unleash the
full potential of network quantization. However, this also results in a
difficult integer programming formulation, and forces most existing approaches
to use an extremely time-consuming search process even with various
relaxations. Instead of solving a problem of the original integer programming,
we propose to optimize a proxy metric, the concept of network orthogonality,
which is highly correlated with the loss of the integer programming but also
easy to optimize with linear programming. This approach reduces the search time
and required data amount by orders of magnitude, with little compromise on
quantization accuracy. Specifically, we achieve 72.08% Top-1 accuracy on
ResNet-18 with 6.7Mb, which does not require any searching iterations. Given
the high efficiency and low data dependency of our algorithm, we used it for
the post-training quantization, which achieve 71.27% Top-1 accuracy on
MobileNetV2 with only 1.5Mb. Our code is available at
https://github.com/MAC-AutoML/OMPQ.
