  We present a novel method for populating 3D indoor scenes with virtual humans
that can navigate in the environment and interact with objects in a realistic
manner. Existing approaches rely on training sequences that contain captured
human motions and the 3D scenes they interact with. However, such interaction
data are costly, difficult to capture, and can hardly cover all plausible
human-scene interactions in complex environments. To address these challenges,
we propose a reinforcement learning-based approach that enables virtual humans
to navigate in 3D scenes and interact with objects realistically and
autonomously, driven by learned motion control policies. The motion control
policies employ latent motion action spaces, which correspond to realistic
motion primitives and are learned from large-scale motion capture data using a
powerful generative motion model. For navigation in a 3D environment, we
propose a scene-aware policy with novel state and reward designs for collision
avoidance. Combined with navigation mesh-based path-finding algorithms to
generate intermediate waypoints, our approach enables the synthesis of diverse
human motions navigating in 3D indoor scenes and avoiding obstacles. To
generate fine-grained human-object interactions, we carefully curate
interaction goal guidance using a marker-based body representation and leverage
features based on the signed distance field (SDF) to encode human-scene
proximity relations. Our method can synthesize realistic and diverse
human-object interactions (e.g.,~sitting on a chair and then getting up) even
for out-of-distribution test scenarios with different object shapes,
orientations, starting body positions, and poses. Experimental results
demonstrate that our approach outperforms state-of-the-art methods in terms of
both motion naturalness and diversity. Code and video results are available at:
https://zkf1997.github.io/DIMOS.
