  We propose learnable polyphase sampling (LPS), a pair of learnable
down/upsampling layers that enable truly shift-invariant and equivariant
convolutional networks. LPS can be trained end-to-end from data and generalizes
existing handcrafted downsampling layers. It is widely applicable as it can be
integrated into any convolutional network by replacing down/upsampling layers.
We evaluate LPS on image classification and semantic segmentation. Experiments
show that LPS is on-par with or outperforms existing methods in both
performance and shift consistency. For the first time, we achieve true
shift-equivariance on semantic segmentation (PASCAL VOC), i.e., 100% shift
consistency, outperforming baselines by an absolute 3.3%.
