  Deep neural networks have been proven effective in a wide range of tasks.
However, their high computational and memory costs make them impractical to
deploy on resource-constrained devices. To address this issue, quantization
schemes have been proposed to reduce the memory footprint and improve inference
speed. While numerous quantization methods have been proposed, they lack
systematic analysis for their effectiveness. To bridge this gap, we collect and
improve existing quantization methods and propose a gold guideline for
post-training quantization. We evaluate the effectiveness of our proposed
method with two popular models, ResNet50 and MobileNetV2, on the ImageNet
dataset. By following our guidelines, no accuracy degradation occurs even after
directly quantizing the model to 8-bits without additional training. A
quantization-aware training based on the guidelines can further improve the
accuracy in lower-bits quantization. Moreover, we have integrated a multi-stage
fine-tuning strategy that works harmoniously with existing pruning techniques
to reduce costs even further. Remarkably, our results reveal that a quantized
MobileNetV2 with 30\% sparsity actually surpasses the performance of the
equivalent full-precision model, underscoring the effectiveness and resilience
of our proposed scheme.
