  Real-world face recognition requires an ability to perceive the unique
features of an individual face across multiple, variable images. The primate
visual system solves the problem of image invariance using cascades of neurons
that convert images of faces into categorical representations of facial
identity. Deep convolutional neural networks (DCNNs) also create generalizable
face representations, but with cascades of simulated neurons. DCNN
representations can be examined in a multidimensional "face space", with
identities and image parameters quantified via their projections onto the axes
that define the space. We examined the organization of viewpoint, illumination,
gender, and identity in this space. We show that the network creates a highly
organized, hierarchically nested, face similarity structure in which
information about face identity and imaging characteristics coexist. Natural
image variation is accommodated in this hierarchy, with face identity nested
under gender, illumination nested under identity, and viewpoint nested under
illumination. To examine identity, we caricatured faces and found that network
identification accuracy increased with caricature level, and--mimicking human
perception--a caricatured distortion of a face "resembled" its veridical
counterpart. Caricatures improved performance by moving the identity away from
other identities in the face space and minimizing the effects of illumination
and viewpoint. Deep networks produce face representations that solve
long-standing computational problems in generalized face recognition. They also
provide a unitary theoretical framework for reconciling decades of behavioral
and neural results that emphasized either the image or the object/face in
representations, without understanding how a neural code could seamlessly
accommodate both.
