  Vision Transformer (ViT) has become one of the most prevailing fundamental
backbone networks in the computer vision community. Despite the high accuracy,
deploying it in real applications raises critical challenges including the high
computational cost and inference latency. Recently, the post-training
quantization (PTQ) technique has emerged as a promising way to enhance ViT's
efficiency. Nevertheless, existing PTQ approaches for ViT suffer from the
inflexible quantization on the post-Softmax and post-GELU activations that obey
the power-law-like distributions. To address these issues, we propose a novel
non-uniform quantizer, dubbed the Adaptive Logarithm AdaLog (AdaLog) quantizer.
It optimizes the logarithmic base to accommodate the power-law-like
distribution of activations, while simultaneously allowing for
hardware-friendly quantization and de-quantization. By employing the bias
reparameterization, the AdaLog quantizer is applicable to both the post-Softmax
and post-GELU activations. Moreover, we develop an efficient Fast Progressive
Combining Search (FPCS) strategy to determine the optimal logarithm base for
AdaLog, as well as the scaling factors and zero points for the uniform
quantizers. Extensive experimental results on public benchmarks demonstrate the
effectiveness of our approach for various ViT-based architectures and vision
tasks including classification, object detection, and instance segmentation.
Code is available at https://github.com/GoatWu/AdaLog.
