  This paper proposes a multi-task learning network with phoneme-aware and
channel-wise attentive learning strategies for text-dependent Speaker
Verification (SV). In the proposed structure, the frame-level multi-task
learning along with the segment-level adversarial learning is adopted for
speaker embedding extraction. The phoneme-aware attentive pooling is exploited
on frame-level features in the main network for speaker classifier, with the
corresponding posterior probability for the phoneme distribution in the
auxiliary subnet. Further, the introduction of Squeeze and Excitation
(SE-block) performs dynamic channel-wise feature recalibration, which improves
the representational ability. The proposed method exploits speaker
idiosyncrasies associated with pass-phrases, and is further improved by the
phoneme-aware attentive pooling and SE-block from temporal and channel-wise
aspects, respectively. The experiments conducted on RSR2015 Part 1 database
confirm that the proposed system achieves outstanding results for textdependent
SV.
