  In deep neural networks (DNNs), there are a huge number of weights and
multiply-and-accumulate (MAC) operations. Accordingly, it is challenging to
apply DNNs on resource-constrained platforms, e.g., mobile phones. Quantization
is a method to reduce the size and the computational complexity of DNNs.
Existing quantization methods either require hardware overhead to achieve a
non-uniform quantization or focus on model-wise and layer-wise uniform
quantization, which are not as fine-grained as filter-wise quantization. In
this paper, we propose a class-based quantization method to determine the
minimum number of quantization bits for each filter or neuron in DNNs
individually. In the proposed method, the importance score of each filter or
neuron with respect to the number of classes in the dataset is first evaluated.
The larger the score is, the more important the filter or neuron is and thus
the larger the number of quantization bits should be. Afterwards, a search
algorithm is adopted to exploit the different importance of filters and neurons
to determine the number of quantization bits of each filter or neuron.
Experimental results demonstrate that the proposed method can maintain the
inference accuracy with low bit-width quantization. Given the same number of
quantization bits, the proposed method can also achieve a better inference
accuracy than the existing methods.
