  As an effective technique to achieve the implementation of deep neural
networks in edge devices, model quantization has been successfully applied in
many practical applications. No matter the methods of quantization aware
training (QAT) or post-training quantization (PTQ), they all depend on the
target bit-widths. When the precision of quantization is adjusted, it is
necessary to fine-tune the quantized model or minimize the quantization noise,
which brings inconvenience in practical applications. In this work, we propose
a method to train a model for all quantization that supports diverse bit-widths
(e.g., form 8-bit to 1-bit) to satisfy the online quantization bit-width
adjustment. It is hot-swappable that can provide specific quantization
strategies for different candidates through multiscale quantization. We use
wavelet decomposition and reconstruction to increase the diversity of weights,
thus significantly improving the performance of each quantization candidate,
especially at ultra-low bit-widths (e.g., 3-bit, 2-bit, and 1-bit).
Experimental results on ImageNet and COCO show that our method can achieve
accuracy comparable performance to dedicated models trained at the same
precision.
