  Naturally controllable human-scene interaction (HSI) generation has an
important role in various fields, such as VR/AR content creation and
human-centered AI. However, existing methods are unnatural and unintuitive in
their controllability, which heavily limits their application in practice.
Therefore, we focus on a challenging task of naturally and controllably
generating realistic and diverse HSIs from textual descriptions. From human
cognition, the ideal generative model should correctly reason about spatial
relationships and interactive actions. To that end, we propose Narrator, a
novel relationship reasoning-based generative approach using a conditional
variation autoencoder for naturally controllable generation given a 3D scene
and a textual description. Also, we model global and local spatial
relationships in a 3D scene and a textual description respectively based on the
scene graph, and introduce a partlevel action mechanism to represent
interactions as atomic body part states. In particular, benefiting from our
relationship reasoning, we further propose a simple yet effective multi-human
generation strategy, which is the first exploration for controllable
multi-human scene interaction generation. Our extensive experiments and
perceptual studies show that Narrator can controllably generate diverse
interactions and significantly outperform existing works. The code and dataset
will be available for research purposes.
