  Deep metric learning is essential for visual recognition. The widely used
pair-wise (or triplet) based loss objectives cannot make full use of semantical
information in training samples or give enough attention to those hard samples
during optimization. Thus, they often suffer from a slow convergence rate and
inferior performance. In this paper, we show how to learn an importance-driven
distance metric via optimal transport programming from batches of samples. It
can automatically emphasize hard examples and lead to significant improvements
in convergence. We propose a new batch-wise optimal transport loss and combine
it in an end-to-end deep metric learning manner. We use it to learn the
distance metric and deep feature representation jointly for recognition.
Empirical results on visual retrieval and classification tasks with six
benchmark datasets, i.e., MNIST, CIFAR10, SHREC13, SHREC14, ModelNet10, and
ModelNet40, demonstrate the superiority of the proposed method. It can
accelerate the convergence rate significantly while achieving a
state-of-the-art recognition performance. For example, in 3D shape recognition
experiments, we show that our method can achieve better recognition performance
within only 5 epochs than what can be obtained by mainstream 3D shape
recognition approaches after 200 epochs.
