  The standard approach to unconstrained face recognition in natural
photographs is via a detection, alignment, recognition pipeline. While that
approach has achieved impressive results, there are several reasons to be
dissatisfied with it, among them is its lack of biological plausibility. A
recent theory of invariant recognition by feedforward hierarchical networks,
like HMAX, other convolutional networks, or possibly the ventral stream,
implies an alternative approach to unconstrained face recognition. This
approach accomplishes detection and alignment implicitly by storing
transformations of training images (called templates) rather than explicitly
detecting and aligning faces at test time. Here we propose a particular
locality-sensitive hashing based voting scheme which we call "consensus of
collisions" and show that it can be used to approximate the full 3-layer
hierarchy implied by the theory. The resulting end-to-end system for
unconstrained face recognition operates on photographs of faces taken under
natural conditions, e.g., Labeled Faces in the Wild (LFW), without aligning or
cropping them, as is normally done. It achieves a drastic improvement in the
state of the art on this end-to-end task, reaching the same level of
performance as the best systems operating on aligned, closely cropped images
(no outside training data). It also performs well on two newer datasets,
similar to LFW, but more difficult: LFW-jittered (new here) and SUFR-W.
