  Automated deception detection (ADD) from real-life videos is a challenging
task. It specifically needs to address two problems: (1) Both face and body
contain useful cues regarding whether a subject is deceptive. How to
effectively fuse the two is thus key to the effectiveness of an ADD model. (2)
Real-life deceptive samples are hard to collect; learning with limited training
data thus challenges most deep learning based ADD models. In this work, both
problems are addressed. Specifically, for face-body multimodal learning, a
novel face-focused cross-stream network (FFCSN) is proposed. It differs
significantly from the popular two-stream networks in that: (a) face detection
is added into the spatial stream to capture the facial expressions explicitly,
and (b) correlation learning is performed across the spatial and temporal
streams for joint deep feature learning across both face and body. To address
the training data scarcity problem, our FFCSN model is trained with both meta
learning and adversarial learning. Extensive experiments show that our FFCSN
model achieves state-of-the-art results. Further, the proposed FFCSN model as
well as its robust training strategy are shown to be generally applicable to
other human-centric video analysis tasks such as emotion recognition from
user-generated videos.
