  Human faces in surveillance videos often suffer from severe image blur,
dramatic pose variations, and occlusion. In this paper, we propose a
comprehensive framework based on Convolutional Neural Networks (CNN) to
overcome challenges in video-based face recognition (VFR). First, to learn
blur-robust face representations, we artificially blur training data composed
of clear still images to account for a shortfall in real-world video training
data. Using training data composed of both still images and artificially
blurred data, CNN is encouraged to learn blur-insensitive features
automatically. Second, to enhance robustness of CNN features to pose variations
and occlusion, we propose a Trunk-Branch Ensemble CNN model (TBE-CNN), which
extracts complementary information from holistic face images and patches
cropped around facial components. TBE-CNN is an end-to-end model that extracts
features efficiently by sharing the low- and middle-level convolutional layers
between the trunk and branch networks. Third, to further promote the
discriminative power of the representations learnt by TBE-CNN, we propose an
improved triplet loss function. Systematic experiments justify the
effectiveness of the proposed techniques. Most impressively, TBE-CNN achieves
state-of-the-art performance on three popular video face databases: PaSC, COX
Face, and YouTube Faces. With the proposed techniques, we also obtain the first
place in the BTAS 2016 Video Person Recognition Evaluation.
