  In the field of text-independent speaker recognition, dynamic models that
adapt along the time axis have been proposed to consider the phoneme-varying
characteristics of speech. However, a detailed analysis of how dynamic models
work depending on phonemes is insufficient. In this paper, we propose temporal
dynamic CNN (TDY-CNN) that considers temporal variation of phonemes by applying
kernels optimally adapting to each time bin. These kernels adapt to time bins
by applying weighted sum of trained basis kernels. Then, an analysis of how
adaptive kernels work on different phonemes in various layers is carried out.
TDY-ResNet-38(x0.5) using six basis kernels improved an equal error rate (EER),
the speaker verification performance, by 17.3% compared to the baseline model
ResNet-38(x0.5). In addition, we showed that adaptive kernels depend on phoneme
groups and are more phoneme-specific at early layers. The temporal dynamic
model adapts itself to phonemes without explicitly given phoneme information
during training, and results show the necessity to consider phoneme variation
within utterances for more accurate and robust text-independent speaker
verification.
