  Despite the success of deep neural networks (DNNs), state-of-the-art models
are too large to deploy on low-resource devices or common server configurations
in which multiple models are held in memory. Model compression methods address
this limitation by reducing the memory footprint, latency, or energy
consumption of a model with minimal impact on accuracy. We focus on the task of
reducing the number of learnable variables in the model. In this work we
combine ideas from weight hashing and dimensionality reductions resulting in a
simple and powerful structured multi-hashing method based on matrix products
that allows direct control of model size of any deep network and is trained
end-to-end. We demonstrate the strength of our approach by compressing models
from the ResNet, EfficientNet, and MobileNet architecture families. Our method
allows us to drastically decrease the number of variables while maintaining
high accuracy. For instance, by applying our approach to EfficentNet-B4 (16M
parameters) we reduce it to to the size of B0 (5M parameters), while gaining
over 3% in accuracy over B0 baseline. On the commonly used benchmark CIFAR10 we
reduce the ResNet32 model by 75% with no loss in quality, and are able to do a
10x compression while still achieving above 90% accuracy.
