  Convolutional neural networks (CNNs) have achieved a great success in face
recognition, which unfortunately comes at the cost of massive computation and
storage consumption. Many compact face recognition networks are thus proposed
to resolve this problem. Triplet loss is effective to further improve the
performance of those compact models. However, it normally employs a fixed
margin to all the samples, which neglects the informative similarity structures
between different identities. In this paper, we propose an enhanced version of
triplet loss, named triplet distillation, which exploits the capability of a
teacher model to transfer the similarity information to a small model by
adaptively varying the margin between positive and negative pairs. Experiments
on LFW, AgeDB, and CPLFW datasets show the merits of our method compared to the
original triplet loss.
