  In this paper, we propose a generalizable mixed-precision quantization (GMPQ)
method for efficient inference. Conventional methods require the consistency of
datasets for bitwidth search and model deployment to guarantee the policy
optimality, leading to heavy search cost on challenging largescale datasets in
realistic applications. On the contrary, our GMPQ searches the
mixed-quantization policy that can be generalized to largescale datasets with
only a small amount of data, so that the search cost is significantly reduced
without performance degradation. Specifically, we observe that locating network
attribution correctly is general ability for accurate visual analysis across
different data distribution. Therefore, despite of pursuing higher model
accuracy and complexity, we preserve attribution rank consistency between the
quantized models and their full-precision counterparts via efficient
capacity-aware attribution imitation for generalizable mixed-precision
quantization strategy search. Extensive experiments show that our method
obtains competitive accuracy-complexity trade-off compared with the
state-of-the-art mixed-precision networks in significantly reduced search cost.
The code is available at https://github.com/ZiweiWangTHU/GMPQ.git.
