  Long-term point tracking is essential to understand non-rigid motion in the
physical world better. Deep learning approaches have recently been incorporated
into long-term point tracking, but most prior work predominantly functions in
2D. Although these methods benefit from the well-established backbones and
matching frameworks, the motions they produce do not always make sense in the
3D physical world. In this paper, we propose the first deep learning framework
for long-term point tracking in 3D that generalizes to new points and videos
without requiring test-time fine-tuning. Our model contains a cost volume
fusion module that effectively integrates multiple past appearances and motion
information via a transformer architecture, significantly enhancing overall
tracking performance. In terms of 3D tracking performance, our model
significantly outperforms simple scene flow chaining and previous 2D point
tracking methods, even if one uses ground truth depth and camera pose to
backproject 2D point tracks in a synthetic scenario.
