  By applying entropy codecs with learned data distributions, neural
compressors have significantly outperformed traditional codecs in terms of
compression ratio. However, the high inference latency of neural networks
hinders the deployment of neural compressors in practical applications. In this
work, we propose Integer-only Discrete Flows (IODF), an efficient neural
compressor with integer-only arithmetic. Our work is built upon integer
discrete flows, which consists of invertible transformations between discrete
random variables. We propose efficient invertible transformations with
integer-only arithmetic based on 8-bit quantization. Our invertible
transformation is equipped with learnable binary gates to remove redundant
filters during inference. We deploy IODF with TensorRT on GPUs, achieving 10x
inference speedup compared to the fastest existing neural compressors, while
retaining the high compression rates on ImageNet32 and ImageNet64.
