  We address the challenge of content diversity and controllability in
pedestrian simulation for driving scenarios. Recent pedestrian animation
frameworks have a significant limitation wherein they primarily focus on either
following trajectory [46] or the content of the reference video [57],
consequently overlooking the potential diversity of human motion within such
scenarios. This limitation restricts the ability to generate pedestrian
behaviors that exhibit a wider range of variations and realistic motions and
therefore restricts its usage to provide rich motion content for other
components in the driving simulation system, e.g., suddenly changed motion to
which the autonomous vehicle should respond. In our approach, we strive to
surpass the limitation by showcasing diverse human motions obtained from
various sources, such as generated human motions, in addition to following the
given trajectory. The fundamental contribution of our framework lies in
combining the motion tracking task with trajectory following, which enables the
tracking of specific motion parts (e.g., upper body) while simultaneously
following the given trajectory by a single policy. This way, we significantly
enhance both the diversity of simulated human motion within the given scenario
and the controllability of the content, including language-based control. Our
framework facilitates the generation of a wide range of human motions,
contributing to greater realism and adaptability in pedestrian simulations for
driving scenarios. More information is on our project page
https://wangjingbo1219.github.io/papers/CVPR2024_PACER_PLUS/PACERPLUSPage.html .
