  As we move closer to real-world AI systems, AI agents must be able to deal
with multiparty (group) conversations. Recognizing and interpreting multiparty
behaviors is challenging, as the system must recognize individual behavioral
cues, deal with the complexity of multiple streams of data from multiple
people, and recognize the subtle contingent social exchanges that take place
amongst group members. To tackle this challenge, we propose the
Multiparty-Transformer (Multipar-T), a transformer model for multiparty
behavior modeling. The core component of our proposed approach is the
Crossperson Attention, which is specifically designed to detect contingent
behavior between pairs of people. We verify the effectiveness of Multipar-T on
a publicly available video-based group engagement detection benchmark, where it
outperforms state-of-the-art approaches in average F-1 scores by 5.2% and
individual class F-1 scores by up to 10.0%. Through qualitative analysis, we
show that our Crossperson Attention module is able to discover contingent
behavior.
