  User-generated cinematic creations are gaining popularity as our daily
entertainment, yet it is a challenge to master cinematography for producing
immersive contents. Many existing automatic methods focus on roughly
controlling predefined shot types or movement patterns, which struggle to
engage viewers with the circumstances of the actor. Real-world cinematographic
rules show that directors can create immersion by comprehensively synchronizing
the camera with the actor. Inspired by this strategy, we propose a deep camera
control framework that enables actor-camera synchronization in three aspects,
considering frame aesthetics, spatial action, and emotional status in the 3D
virtual stage. Following rule-of-thirds, our framework first modifies the
initial camera placement to position the actor aesthetically. This adjustment
is facilitated by a self-supervised adjustor that analyzes frame composition
via camera projection. We then design a GAN model that can adversarially
synthesize fine-grained camera movement based on the physical action and
psychological state of the actor, using an encoder-decoder generator to map
kinematics and emotional variables into camera trajectories. Moreover, we
incorporate a regularizer to align the generated stylistic variances with
specific emotional categories and intensities. The experimental results show
that our proposed method yields immersive cinematic videos of high quality,
both quantitatively and qualitatively. Live examples can be found in the
supplementary video.
