  We present the Deception Detection and Physiological Monitoring (DDPM)
dataset and initial baseline results on this dataset. Our application context
is an interview scenario in which the interviewee attempts to deceive the
interviewer on selected responses. The interviewee is recorded in RGB,
near-infrared, and long-wave infrared, along with cardiac pulse, blood
oxygenation, and audio. After collection, data were annotated for
interviewer/interviewee, curated, ground-truthed, and organized into train /
test parts for a set of canonical deception detection experiments. Baseline
experiments found random accuracy for micro-expressions as an indicator of
deception, but that saccades can give a statistically significant response. We
also estimated subject heart rates from face videos (remotely) with a mean
absolute error as low as 3.16 bpm. The database contains almost 13 hours of
recordings of 70 subjects, and over 8 million visible-light, near-infrared, and
thermal video frames, along with appropriate meta, audio and pulse oximeter
data. To our knowledge, this is the only collection offering recordings of five
modalities in an interview scenario that can be used in both deception
detection and remote photoplethysmography research.
