  Human-Object Interaction (HOI) detection is a fundamental visual task aiming
at localizing and recognizing interactions between humans and objects. Existing
works focus on the visual and linguistic features of humans and objects.
However, they do not capitalise on the high-level and semantic relationships
present in the image, which provides crucial contextual and detailed relational
knowledge for HOI inference. We propose a novel method to exploit this
information, through the scene graph, for the Human-Object Interaction (SG2HOI)
detection task. Our method, SG2HOI, incorporates the SG information in two
ways: (1) we embed a scene graph into a global context clue, serving as the
scene-specific environmental context; and (2) we build a relation-aware
message-passing module to gather relationships from objects' neighborhood and
transfer them into interactions. Empirical evaluation shows that our SG2HOI
method outperforms the state-of-the-art methods on two benchmark HOI datasets:
V-COCO and HICO-DET. Code will be available at https://github.com/ht014/SG2HOI.
