  We present an efficient alternative to the convolutional layer using cheap
spatial transformations. This construction exploits an inherent spatial
redundancy of the learned convolutional filters to enable a much greater
parameter efficiency, while maintaining the top-end accuracy of their dense
counter-parts. Training these networks is modelled as a generalised pruning
problem, whereby the pruned filters are replaced with cheap transformations
from the set of non-pruned filters. We provide an efficient implementation of
the proposed layer, followed by two natural extensions to avoid excessive
feature compression and to improve the expressivity of the transformed
features. We show that these networks can achieve comparable or improved
performance to state-of-the-art pruning models across both the CIFAR-10 and
ImageNet-1K datasets.
