  In deep metric learning, the Triplet Loss has emerged as a popular method to
learn many computer vision and natural language processing tasks such as facial
recognition, object detection, and visual-semantic embeddings. One issue that
plagues the Triplet Loss is network collapse, an undesirable phenomenon where
the network projects the embeddings of all data onto a single point.
Researchers predominately solve this problem by using triplet mining
strategies. While hard negative mining is the most effective of these
strategies, existing formulations lack strong theoretical justification for
their empirical success. In this paper, we utilize the mathematical theory of
isometric approximation to show an equivalence between the Triplet Loss sampled
by hard negative mining and an optimization problem that minimizes a
Hausdorff-like distance between the neural network and its ideal counterpart
function. This provides the theoretical justifications for hard negative
mining's empirical efficacy. In addition, our novel application of the
isometric approximation theorem provides the groundwork for future forms of
hard negative mining that avoid network collapse. Our theory can also be
extended to analyze other Euclidean space-based metric learning methods like
Ladder Loss or Contrastive Learning.
