  Controllable human motion synthesis is essential for applications in AR/VR,
gaming and embodied AI. Existing methods often focus solely on either language
or full trajectory control, lacking precision in synthesizing motions aligned
with user-specified trajectories, especially for multi-joint control. To
address these issues, we present TLControl, a novel method for realistic human
motion synthesis, incorporating both low-level Trajectory and high-level
Language semantics controls, through the integration of neural-based and
optimization-based techniques. Specifically, we begin with training a VQ-VAE
for a compact and well-structured latent motion space organized by body parts.
We then propose a Masked Trajectories Transformer (MTT) for predicting a motion
distribution conditioned on language and trajectory. Once trained, we use MTT
to sample initial motion predictions given user-specified partial trajectories
and text descriptions as conditioning. Finally, we introduce a test-time
optimization to refine these coarse predictions for precise trajectory control,
which offers flexibility by allowing users to specify various optimization
goals and ensures high runtime efficiency. Comprehensive experiments show that
TLControl significantly outperforms the state-of-the-art in trajectory accuracy
and time efficiency, making it practical for interactive and high-quality
animation generation.
