  Generating informative scene graphs from images requires integrating and
reasoning from various graph components, i.e., objects and relationships.
However, current scene graph generation (SGG) methods, including the unbiased
SGG methods, still struggle to predict informative relationships due to the
lack of 1) high-level inference such as transitive inference between
relationships and 2) efficient mechanisms that can incorporate all interactions
of graph components. To address the issues mentioned above, we devise a
hyper-relationship learning network, termed HLN, for SGG. Specifically, the
proposed HLN stems from hypergraphs and two graph attention networks (GATs) are
designed to infer relationships: 1) the object-relationship GAT or OR-GAT to
explore interactions between objects and relationships, and 2) the
hyper-relationship GAT or HR-GAT to integrate transitive inference of
hyper-relationships, i.e., the sequential relationships between three objects
for transitive reasoning. As a result, HLN significantly improves the
performance of scene graph generation by integrating and reasoning from object
interactions, relationship interactions, and transitive inference of
hyper-relationships. We evaluate HLN on the most popular SGG dataset, i.e., the
Visual Genome dataset, and the experimental results demonstrate its great
superiority over recent state-of-the-art methods. For example, the proposed HLN
improves the recall per relationship from 11.3\% to 13.1\%, and maintains the
recall per image from 19.8\% to 34.9\%. We will release the source code and
pretrained models on GitHub.
