  As an important and challenging problem in computer vision and graphics,
keypoint-based object tracking is typically formulated in a spatio-temporal
statistical learning framework. However, most existing keypoint trackers are
incapable of effectively modeling and balancing the following three aspects in
a simultaneous manner: temporal model coherence across frames, spatial model
consistency within frames, and discriminative feature construction. To address
this issue, we propose a robust keypoint tracker based on spatio-temporal
multi-task structured output optimization driven by discriminative metric
learning. Consequently, temporal model coherence is characterized by multi-task
structured keypoint model learning over several adjacent frames, while spatial
model consistency is modeled by solving a geometric verification based
structured learning problem. Discriminative feature construction is enabled by
metric learning to ensure the intra-class compactness and inter-class
separability. Finally, the above three modules are simultaneously optimized in
a joint learning scheme. Experimental results have demonstrated the
effectiveness of our tracker.
