  Graph convolutional networks (GCNs) have been the predominant methods in
skeleton-based human action recognition, including human-human interaction
recognition. However, when dealing with interaction sequences, current
GCN-based methods simply split the two-person skeleton into two discrete graphs
and perform graph convolution separately as done for single-person action
classification. Such operations ignore rich interactive information and hinder
effective spatial inter-body relationship modeling. To overcome the above
shortcoming, we introduce a novel unified two-person graph to represent
inter-body and intra-body correlations between joints. Experiments show
accuracy improvements in recognizing both interactions and individual actions
when utilizing the proposed two-person graph topology. In addition, We design
several graph labeling strategies to supervise the model to learn discriminant
spatial-temporal interactive features. Finally, we propose a two-person graph
convolutional network (2P-GCN). Our model achieves state-of-the-art results on
four benchmarks of three interaction datasets: SBU, interaction subsets of
NTU-RGB+D and NTU-RGB+D 120.
