  Compared with global average pooling in existing deep convolutional neural
networks (CNNs), global covariance pooling can capture richer statistics of
deep features, having potential for improving representation and generalization
abilities of deep CNNs. However, integration of global covariance pooling into
deep CNNs brings two challenges: (1) robust covariance estimation given deep
features of high dimension and small sample size; (2) appropriate usage of
geometry of covariances. To address these challenges, we propose a global
Matrix Power Normalized COVariance (MPN-COV) Pooling. Our MPN-COV conforms to a
robust covariance estimator, very suitable for scenario of high dimension and
small sample size. It can also be regarded as Power-Euclidean metric between
covariances, effectively exploiting their geometry. Furthermore, a global
Gaussian embedding network is proposed to incorporate first-order statistics
into MPN-COV. For fast training of MPN-COV networks, we implement an iterative
matrix square root normalization, avoiding GPU unfriendly eigen-decomposition
inherent in MPN-COV. Additionally, progressive 1x1 convolutions and group
convolution are introduced to compress covariance representations. The proposed
methods are highly modular, readily plugged into existing deep CNNs. Extensive
experiments are conducted on large-scale object classification, scene
categorization, fine-grained visual recognition and texture classification,
showing our methods outperform the counterparts and obtain state-of-the-art
performance.
