  Face recognition techniques have been developed significantly in recent
years. However, recognizing faces with partial occlusion is still challenging
for existing face recognizers which is heavily desired in real-world
applications concerning surveillance and security. Although much research
effort has been devoted to developing face de-occlusion methods, most of them
can only work well under constrained conditions, such as all the faces are from
a pre-defined closed set. In this paper, we propose a robust LSTM-Autoencoders
(RLA) model to effectively restore partially occluded faces even in the wild.
The RLA model consists of two LSTM components, which aims at occlusion-robust
face encoding and recurrent occlusion removal respectively. The first one,
named multi-scale spatial LSTM encoder, reads facial patches of various scales
sequentially to output a latent representation, and occlusion-robustness is
achieved owing to the fact that the influence of occlusion is only upon some of
the patches. Receiving the representation learned by the encoder, the LSTM
decoder with a dual channel architecture reconstructs the overall face and
detects occlusion simultaneously, and by feat of LSTM, the decoder breaks down
the task of face de-occlusion into restoring the occluded part step by step.
Moreover, to minimize identify information loss and guarantee face recognition
accuracy over recovered faces, we introduce an identity-preserving adversarial
training scheme to further improve RLA. Extensive experiments on both synthetic
and real datasets of faces with occlusion clearly demonstrate the effectiveness
of our proposed RLA in removing different types of facial occlusion at various
locations. The proposed method also provides significantly larger performance
gain than other de-occlusion methods in promoting recognition performance over
partially-occluded faces.
