  In human-object interactions (HOI) recognition, conventional methods consider
the human body as a whole and pay a uniform attention to the entire body
region. They ignore the fact that normally, human interacts with an object by
using some parts of the body. In this paper, we argue that different body parts
should be paid with different attention in HOI recognition, and the
correlations between different body parts should be further considered. This is
because our body parts always work collaboratively. We propose a new pairwise
body-part attention model which can learn to focus on crucial parts, and their
correlations for HOI recognition. A novel attention based feature selection
method and a feature representation scheme that can capture pairwise
correlations between body parts are introduced in the model. Our proposed
approach achieved 4% improvement over the state-of-the-art results in HOI
recognition on the HICO dataset. We will make our model and source codes
publicly available.
