  Detecting groups of people who are jointly deceptive in video conversations
is crucial in settings such as meetings, sales pitches, and negotiations. Past
work on deception in videos focuses on detecting a single deceiver and uses
facial or visual features only. In this paper, we propose the concept of
Face-to-Face Dynamic Interaction Networks (FFDINs) to model the interpersonal
interactions within a group of people. The use of FFDINs enables us to leverage
network relations in detecting group deception in video conversations for the
first time. We use a dataset of 185 videos from a deception-based game called
Resistance. We first characterize the behavior of individual, pairs, and groups
of deceptive participants and compare them to non-deceptive participants. Our
analysis reveals that pairs of deceivers tend to avoid mutual interaction and
focus their attention on non-deceivers. In contrast, non-deceivers interact
with everyone equally. We propose Negative Dynamic Interaction Networks to
capture the notion of missing interactions. We create the DeceptionRank
algorithm to detect deceivers from NDINs extracted from videos that are just
one minute long. We show that our method outperforms recent state-of-the-art
computer vision, graph embedding, and ensemble methods by at least 20.9% AUROC
in identifying deception from videos.
