  Creating scenes for captured motions that achieve realistic human-scene
interaction is crucial for 3D animation in movies or video games. As character
motion is often captured in a blue-screened studio without real furniture or
objects in place, there may be a discrepancy between the planned motion and the
captured one. This gives rise to the need for automatic scene layout generation
to relieve the burdens of selecting and positioning furniture and objects.
Previous approaches cannot avoid artifacts like penetration and floating due to
the lack of physical constraints. Furthermore, some heavily rely on specific
data to learn the contact affordances, restricting the generalization ability
to different motions. In this work, we present a physics-based approach that
simultaneously optimizes a scene layout generator and simulates a moving human
in a physics simulator. To attain plausible and realistic interaction motions,
our method explicitly introduces physical constraints. To automatically recover
and generate the scene layout, we minimize the motion tracking errors to
identify the objects that can afford interaction. We use reinforcement learning
to perform a dual-optimization of both the character motion imitation
controller and the scene layout generator. To facilitate the optimization, we
reshape the tracking rewards and devise pose prior guidance obtained from our
estimated pseudo-contact labels. We evaluate our method using motions from SAMP
and PROX, and demonstrate physically plausible scene layout reconstruction
compared with the previous kinematics-based method.
