  This work targets the commonly used FPGA (field-programmable gate array)
devices as the hardware platform for DNN edge computing. We focus on DNN
quantization as the main model compression technique. The novelty of this work
is: We use a quantization method that supports multiple precisions along the
intra-layer dimension, while the existing quantization methods apply
multi-precision quantization along the inter-layer dimension. The intra-layer
multi-precision method can uniform the hardware configurations for different
layers to reduce computation overhead and at the same time preserve the model
accuracy as the inter-layer approach. Our proposed ILMPQ DNN quantization
framework achieves 70.73 Top1 accuracy in ResNet-18 on the ImageNet dataset. We
also validate the proposed MSP framework on two FPGA devices i.e., Xilinx
XC7Z020 and XC7Z045. We achieve 3.65x speedup in end-to-end inference time on
the ImageNet, compared with the fixed-point quantization method.
