  This paper proposes an additive phoneme-aware margin softmax (APM-Softmax)
loss to train the multi-task learning network with phonetic information for
language recognition. In additive margin softmax (AM-Softmax) loss, the margin
is set as a constant during the entire training for all training samples, and
that is a suboptimal method since the recognition difficulty varies in training
samples. In additive angular margin softmax (AAM-Softmax) loss, the additional
angular margin is set as a costant as well. In this paper, we propose an
APM-Softmax loss for language recognition with phoneitc multi-task learning, in
which the additive phoneme-aware margin is automatically tuned for different
training samples. More specifically, the margin of language recognition is
adjusted according to the results of phoneme recognition. Experiments are
reported on Oriental Language Recognition (OLR) datasets, and the proposed
method improves AM-Softmax loss and AAM-Softmax loss in different language
recognition testing conditions.
