  We introduce Eigen Evolution Pooling, an efficient method to aggregate a
sequence of feature vectors. Eigen evolution pooling is designed to produce
compact feature representations for a sequence of feature vectors, while
maximally preserving as much information about the sequence as possible,
especially the temporal evolution of the features over time. Eigen evolution
pooling is a general pooling method that can be applied to any sequence of
feature vectors, from low-level RGB values to high-level Convolutional Neural
Network (CNN) feature vectors. We show that eigen evolution pooling is more
effective than average, max, and rank pooling for encoding the dynamics of
human actions in video. We demonstrate the power of eigen evolution pooling on
UCF101 and Hollywood2 datasets, two human action recognition benchmarks, and
achieve state-of-the-art performance.
