  Compared with the progress made on human activity classification, much less
success has been achieved on human interaction understanding (HIU). Apart from
the latter task is much more challenging, the main cause is that recent
approaches learn human interactive relations via shallow graphical models,
which is inadequate to model complicated human interactions. In this paper, we
propose a consistency-aware graph network, which combines the representative
ability of graph network and the consistency-aware reasoning to facilitate the
HIU task. Our network consists of three components, a backbone CNN to extract
image features, a factor graph network to learn third-order interactive
relations among participants, and a consistency-aware reasoning module to
enforce labeling and grouping consistencies. Our key observation is that the
consistency-aware-reasoning bias for HIU can be embedded into an energy
function, minimizing which delivers consistent predictions. An efficient
mean-field inference algorithm is proposed, such that all modules of our
network could be trained jointly in an end-to-end manner. Experimental results
show that our approach achieves leading performance on three benchmarks.
