  Estimating 3D poses of multiple humans in real-time is a classic but still
challenging task in computer vision. Its major difficulty lies in the ambiguity
in cross-view association of 2D poses and the huge state space when there are
multiple people in multiple views. In this paper, we present a novel solution
for multi-human 3D pose estimation from multiple calibrated camera views. It
takes 2D poses in different camera coordinates as inputs and aims for the
accurate 3D poses in the global coordinate. Unlike previous methods that
associate 2D poses among all pairs of views from scratch at every frame, we
exploit the temporal consistency in videos to match the 2D inputs with 3D poses
directly in 3-space. More specifically, we propose to retain the 3D pose for
each person and update them iteratively via the cross-view multi-human
tracking. This novel formulation improves both accuracy and efficiency, as we
demonstrated on widely-used public datasets. To further verify the scalability
of our method, we propose a new large-scale multi-human dataset with 12 to 28
camera views. Without bells and whistles, our solution achieves 154 FPS on 12
cameras and 34 FPS on 28 cameras, indicating its ability to handle large-scale
real-world applications. The proposed dataset is released at
https://github.com/longcw/crossview_3d_pose_tracking.
