  Face representation is a crucial step of face recognition systems. An optimal
face representation should be discriminative, robust, compact, and very
easy-to-implement. While numerous hand-crafted and learning-based
representations have been proposed, considerable room for improvement is still
present. In this paper, we present a very easy-to-implement deep learning
framework for face representation. Our method bases on a new structure of deep
network (called Pyramid CNN). The proposed Pyramid CNN adopts a
greedy-filter-and-down-sample operation, which enables the training procedure
to be very fast and computation-efficient. In addition, the structure of
Pyramid CNN can naturally incorporate feature sharing across multi-scale face
representations, increasing the discriminative ability of resulting
representation. Our basic network is capable of achieving high recognition
accuracy ($85.8\%$ on LFW benchmark) with only 8 dimension representation. When
extended to feature-sharing Pyramid CNN, our system achieves the
state-of-the-art performance ($97.3\%$) on LFW benchmark. We also introduce a
new benchmark of realistic face images on social network and validate our
proposed representation has a good ability of generalization.
