  Automatic pain recognition is paramount for medical diagnosis and treatment.
The existing works fall into three categories: assessing facial appearance
changes, exploiting physiological cues, or fusing them in a multi-modal manner.
However, (1) appearance changes are easily affected by subjective factors which
impedes objective pain recognition. Besides, the appearance-based approaches
ignore long-range spatial-temporal dependencies that are important for modeling
expressions over time; (2) the physiological cues are obtained by attaching
sensors on human body, which is inconvenient and uncomfortable. In this paper,
we present a novel multi-task learning framework which encodes both appearance
changes and physiological cues in a non-contact manner for pain recognition.
The framework is able to capture both local and long-range dependencies via the
proposed attention mechanism for the learned appearance representations, which
are further enriched by temporally attended physiological cues (remote
photoplethysmography, rPPG) that are recovered from videos in the auxiliary
task. This framework is dubbed rPPG-enriched Spatio-Temporal Attention Network
(rSTAN) and allows us to establish the state-of-the-art performance of
non-contact pain recognition on publicly available pain databases. It
demonstrates that rPPG predictions can be used as an auxiliary task to
facilitate non-contact automatic pain recognition.
