  In many application domains such as computer vision, Convolutional Layers
(CLs) are key to the accuracy of deep learning methods. However, it is often
required to assemble a large number of CLs, each containing thousands of
parameters, in order to reach state-of-the-art accuracy, thus resulting in
complex and demanding systems that are poorly fitted to resource-limited
devices. Recently, methods have been proposed to replace the generic
convolution operator by the combination of a shift operation and a simpler 1x1
convolution. The resulting block, called Shift Layer (SL), is an efficient
alternative to CLs in the sense it allows to reach similar accuracies on
various tasks with faster computations and fewer parameters. In this
contribution, we introduce Shift Attention Layers (SALs), which extend SLs by
using an attention mechanism that learns which shifts are the best at the same
time the network function is trained. We demonstrate SALs are able to
outperform vanilla SLs (and CLs) on various object recognition benchmarks while
significantly reducing the number of float operations and parameters for the
inference.
