  Recently, pre-trained language models like BERT have shown promising
performance on multiple natural language processing tasks. However, the
application of these models has been limited due to their huge size. To reduce
its size, a popular and efficient way is quantization. Nevertheless, most of
the works focusing on BERT quantization adapted primary linear clustering as
the quantization scheme, and few works try to upgrade it. That limits the
performance of quantization significantly. In this paper, we implement k-means
quantization and compare its performance on the fix-precision quantization of
BERT with linear quantization. Through the comparison, we verify that the
effect of the underlying quantization scheme upgrading is underestimated and
there is a huge development potential of k-means quantization. Besides, we also
compare the two quantization schemes on ALBERT models to explore the robustness
differences between different pre-trained models.
