  3D visual tracking is significant to deep space exploration programs, which
can guarantee spacecraft to flexibly approach the target. In this paper, we
focus on the studied accurate and real-time method for 3D tracking. Considering
the fact that there are almost no public dataset for this topic, A new
large-scale 3D asteroid tracking dataset is presented, including binocular
video sequences, depth maps, and point clouds of diverse asteroids with various
shapes and textures. Benefitting from the power and convenience of simulation
platform, all the 2D and 3D annotations are automatically generated. Meanwhile,
we propose a deep-learning based 3D tracking framework, named as Track3D, which
involves 2D monocular tracker and a novel light-weight amodal axis-aligned
bounding-box network, A3BoxNet. The evaluation results demonstrate that Track3D
achieves state-of-the-art 3D tracking performance in both accuracy and
precision, comparing to a baseline algorithm. Moreover, our framework has great
generalization ability to 2D monocular tracking performance.
