  Human behavior understanding requires looking at minute details in the large
context of a scene containing multiple input modalities. It is necessary as it
allows the design of more human-like machines. While transformer approaches
have shown great improvements, they face multiple challenges such as lack of
data or background noise. To tackle these, we introduce the Forced Attention
(FAt) Transformer which utilize forced attention with a modified backbone for
input encoding and a use of additional inputs. In addition to improving the
performance on different tasks and inputs, the modification requires less time
and memory resources. We provide a model for a generalised feature extraction
for tasks concerning social signals and behavior analysis. Our focus is on
understanding behavior in videos where people are interacting with each other
or talking into the camera which simulates the first person point of view in
social interaction. FAt Transformers are applied to two downstream tasks:
personality recognition and body language recognition. We achieve
state-of-the-art results for Udiva v0.5, First Impressions v2 and MPII Group
Interaction datasets. We further provide an extensive ablation study of the
proposed architecture.
