  Power Normalizations (PN) are very useful non-linear operators in the context
of Bag-of-Words data representations as they tackle problems such as feature
imbalance. In this paper, we reconsider these operators in the deep learning
setup by introducing a novel layer that implements PN for non-linear pooling of
feature maps. Specifically, by using a kernel formulation, our layer combines
the feature vectors and their respective spatial locations in the feature maps
produced by the last convolutional layer of CNN. Linearization of such a kernel
results in a positive definite matrix capturing the second-order statistics of
the feature vectors, to which PN operators are applied. We study two types of
PN functions, namely (i) MaxExp and (ii) Gamma, addressing their role and
meaning in the context of nonlinear pooling. We also provide a probabilistic
interpretation of these operators and derive their surrogates with well-behaved
gradients for end-to-end CNN learning. We apply our theory to practice by
implementing the PN layer on a ResNet-50 model and showcase experiments on four
benchmarks for fine-grained recognition, scene recognition, and material
classification. Our results demonstrate state-of-the-art performance across all
these tasks.
