  Human action recognition (HAR) in videos is one of the core tasks of video
understanding. Based on video sequences, the goal is to recognize actions
performed by humans. While HAR has received much attention in the visible
spectrum, action recognition in infrared videos is little studied. Accurate
recognition of human actions in the infrared domain is a highly challenging
task because of the redundant and indistinguishable texture features present in
the sequence. Furthermore, in some cases, challenges arise from the irrelevant
information induced by the presence of multiple active persons not contributing
to the actual action of interest. Therefore, most existing methods consider a
standard paradigm that does not take into account these challenges, which is in
some part due to the ambiguous definition of the recognition task in some
cases. In this paper, we propose a new method that simultaneously learns to
recognize efficiently human actions in the infrared spectrum, while
automatically identifying the key-actors performing the action without using
any prior knowledge or explicit annotations. Our method is composed of three
stages. In the first stage, optical flow-based key-actor identification is
performed. Then for each key-actor, we estimate key-poses that will guide the
frame selection process. A scale-invariant encoding process along with embedded
pose filtering are performed in order to enhance the quality of action
representations. Experimental results on InfAR dataset show that our proposed
model achieves promising recognition performance and learns useful action
representations.
