  Distance metric learning (DML) is to learn the embeddings where examples from
the same class are closer than examples from different classes. It can be cast
as an optimization problem with triplet constraints. Due to the vast number of
triplet constraints, a sampling strategy is essential for DML. With the
tremendous success of deep learning in classifications, it has been applied for
DML. When learning embeddings with deep neural networks (DNNs), only a
mini-batch of data is available at each iteration. The set of triplet
constraints has to be sampled within the mini-batch. Since a mini-batch cannot
capture the neighbors in the original set well, it makes the learned embeddings
sub-optimal. On the contrary, optimizing SoftMax loss, which is a
classification loss, with DNN shows a superior performance in certain DML
tasks. It inspires us to investigate the formulation of SoftMax. Our analysis
shows that SoftMax loss is equivalent to a smoothed triplet loss where each
class has a single center. In real-world data, one class can contain several
local clusters rather than a single one, e.g., birds of different poses.
Therefore, we propose the SoftTriple loss to extend the SoftMax loss with
multiple centers for each class. Compared with conventional deep metric
learning algorithms, optimizing SoftTriple loss can learn the embeddings
without the sampling phase by mildly increasing the size of the last fully
connected layer. Experiments on the benchmark fine-grained data sets
demonstrate the effectiveness of the proposed loss function. Code is available
at https://github.com/idstcv/SoftTriple
