  In modern computer vision tasks, convolutional neural networks (CNNs) are
indispensable for image classification tasks due to their efficiency and
effectiveness. Part of their superiority compared to other architectures, comes
from the fact that a single, local filter is shared across the entire image.
However, there are scenarios where we may need to treat spatial locations in
non-uniform manner. We see this in nature when considering how humans have
evolved foveation to process different areas in their field of vision with
varying levels of detail. In this paper we propose a way to enable CNNs to
learn different pooling weights for each pixel location. We do so by
introducing an extended definition of a pooling operator. This operator can
learn a strict super-set of what can be learned by average pooling or
convolutions. It has the benefit of being shared across feature maps and can be
encouraged to be local or diffuse depending on the data. We show that for fixed
network weights, our pooling operator can be computed in closed-form by
spectral decomposition of matrices associated with class separability. Through
experiments, we show that this operator benefits generalization for ResNets and
CNNs on the CIFAR-10, CIFAR-100 and SVHN datasets and improves robustness to
geometric corruptions and perturbations on the CIFAR-10-C and CIFAR-10-P test
sets.
