  Recent scene graph generation (SGG) frameworks have focused on learning
complex relationships among multiple objects in an image. Thanks to the nature
of the message passing neural network (MPNN) that models high-order
interactions between objects and their neighboring objects, they are dominant
representation learning modules for SGG. However, existing MPNN-based
frameworks assume the scene graph as a homogeneous graph, which restricts the
context-awareness of visual relations between objects. That is, they overlook
the fact that the relations tend to be highly dependent on the objects with
which the relations are associated. In this paper, we propose an unbiased
heterogeneous scene graph generation (HetSGG) framework that captures
relation-aware context using message passing neural networks. We devise a novel
message passing layer, called relation-aware message passing neural network
(RMP), that aggregates the contextual information of an image considering the
predicate type between objects. Our extensive evaluations demonstrate that
HetSGG outperforms state-of-the-art methods, especially outperforming on tail
predicate classes.
