  This paper explores the combination of neural network quantization and
entropy coding for memory footprint minimization. Edge deployment of quantized
models is hampered by the harsh Pareto frontier of the accuracy-to-bitwidth
tradeoff, causing dramatic accuracy loss below a certain bitwidth. This
accuracy loss can be alleviated thanks to mixed precision quantization,
allowing for more flexible bitwidth allocation. However, standard mixed
precision benefits remain limited due to the 1-bit frontier, that forces each
parameter to be encoded on at least 1 bit of data. This paper introduces an
approach that combines mixed precision, zero-point quantization and entropy
coding to push the compression boundary of Resnets beyond the 1-bit frontier
with an accuracy drop below 1% on the ImageNet benchmark. From an
implementation standpoint, a compact decoder architecture features reduced
latency, thus allowing for inference-compatible decoding.
