  In this paper, we propose a novel Siamese Adversarial Network (SAN)
architecture for automatic speech recognition, which aims at solving the
difficulty of fuzzy audio recognition. Specifically, SAN constructs two
sub-networks to differentiate the audio feature input and then introduces a
loss to unify the output distribution of these sub-networks. Adversarial
learning enables the network to capture more essential acoustic features and
helps the models achieve better performance when encountering fuzzy audio
input. We conduct numerical experiments with the SAN model on several datasets
for the automatic speech recognition task. All experimental results show that
the siamese adversarial nets significantly reduce the character error rate
(CER). Specifically, we achieve a new state of art 4.37 CER without language
model on the AISHELL-1 dataset, which leads to around 5% relative CER
reduction. To reveal the generality of the siamese adversarial net, we also
conduct experiments on the phoneme recognition task, which also shows the
superiority of the siamese adversarial network.
