  High-quality motion reconstruction that follows the user's movements can be
achieved by high-end mocap systems with many sensors. However, obtaining such
animation quality with fewer input devices is gaining popularity as it brings
mocap closer to the general public. The main challenges include the loss of
end-effector accuracy in learning-based approaches, or the lack of naturalness
and smoothness in IK-based solutions. In addition, such systems are often
finely tuned to a specific number of trackers and are highly sensitive to
missing data e.g., in scenarios where a sensor is occluded or malfunctions. In
response to these challenges, we introduce DragPoser, a novel
deep-learning-based motion reconstruction system that accurately represents
hard and dynamic on-the-fly constraints, attaining real-time high end-effectors
position accuracy. This is achieved through a pose optimization process within
a structured latent space. Our system requires only one-time training on a
large human motion dataset, and then constraints can be dynamically defined as
losses, while the pose is iteratively refined by computing the gradients of
these losses within the latent space. To further enhance our approach, we
incorporate a Temporal Predictor network, which employs a Transformer
architecture to directly encode temporality within the latent space. This
network ensures the pose optimization is confined to the manifold of valid
poses and also leverages past pose data to predict temporally coherent poses.
Results demonstrate that DragPoser surpasses both IK-based and the latest
data-driven methods in achieving precise end-effector positioning, while it
produces natural poses and temporally coherent motion. In addition, our system
showcases robustness against on-the-fly constraint modifications, and exhibits
exceptional adaptability to various input configurations and changes.
