  Skeleton-based two-person interaction recognition has been gaining increasing
attention as advancements are made in pose estimation and graph convolutional
networks. Although the accuracy has been gradually improving, the increasing
computational complexity makes it more impractical for a real-world
environment. There is still room for accuracy improvement as the conventional
methods do not fully represent the relationship between inter-body joints. In
this paper, we propose a lightweight model for accurately recognizing
two-person interactions. In addition to the architecture, which incorporates
middle fusion, we introduce a factorized convolution technique to reduce the
weight parameters of the model. We also introduce a network stream that
accounts for relative distance changes between inter-body joints to improve
accuracy. Experiments using two large-scale datasets, NTU RGB+D 60 and 120,
show that our method simultaneously achieved the highest accuracy and
relatively low computational complexity compared with the conventional methods.
