  Automatic face recognition has received significant performance improvement
by developing specialised facial image representations. On the other hand,
generic object recognition has rarely been applied to the face recognition.
Spatial pyramid pooling of features encoded by an over-complete dictionary has
been the key component of many state-of-the-art image classification systems.
Inspired by its success, in this work we develop a new face image
representation method inspired by the second-order pooling in Carreira et al.
[1], which was originally proposed for image segmentation. The proposed method
differs from the previous methods in that, we encode the densely extracted
local patches by a small-size dictionary; and the facial image signatures are
obtained by pooling the second-order statistics of the encoded features. We
show the importance of pooling on encoded features, which is bypassed by the
original second-order pooling method to avoid the high computational cost.
Equipped with a simple linear classifier, the proposed method outperforms the
state-of-the-art face identification performance by large margins. For example,
on the LFW databases, the proposed method performs better than the previous
best by around 13% accuracy.
