  Social relationships (e.g., friends, couple etc.) form the basis of the
social network in our daily life. Automatically interpreting such relationships
bears a great potential for the intelligent systems to understand human
behavior in depth and to better interact with people at a social level. Human
beings interpret the social relationships within a group not only based on the
people alone, and the interplay between such social relationships and the
contextual information around the people also plays a significant role.
However, these additional cues are largely overlooked by the previous studies.
We found that the interplay between these two factors can be effectively
modeled by a novel structured knowledge graph with proper message propagation
and attention. And this structured knowledge can be efficiently integrated into
the deep neural network architecture to promote social relationship
understanding by an end-to-end trainable Graph Reasoning Model (GRM), in which
a propagation mechanism is learned to propagate node message through the graph
to explore the interaction between persons of interest and the contextual
objects. Meanwhile, a graph attentional mechanism is introduced to explicitly
reason about the discriminative objects to promote recognition. Extensive
experiments on the public benchmarks demonstrate the superiority of our method
over the existing leading competitors.
