  Existing face recognition using deep neural networks is difficult to know
what kind of features are used to discriminate the identities of face images
clearly. To investigate the effective features for face recognition, we propose
a novel face recognition method, called a pairwise relational network (PRN),
that obtains local appearance patches around landmark points on the feature
map, and captures the pairwise relation between a pair of local appearance
patches. The PRN is trained to capture unique and discriminative pairwise
relations among different identities. Because the existence and meaning of
pairwise relations should be identity dependent, we add a face identity state
feature, which obtains from the long short-term memory (LSTM) units network
with the sequential local appearance patches on the feature maps, to the PRN.
To further improve accuracy of face recognition, we combined the global
appearance representation with the pairwise relational feature. Experimental
results on the LFW show that the PRN using only pairwise relations achieved
99.65% accuracy and the PRN using both pairwise relations and face identity
state feature achieved 99.76% accuracy. On the YTF, both the PRN using only
pairwise relations and the PRN using pairwise relations and the face identity
state feature achieved the state-of-the-art (95.7% and 96.3%). The PRN also
achieved comparable results to the state-of-the-art for both face verification
and face identification tasks on the IJB-A, and the state-of-the-art on the
IJB-B.
