  Environmental noises and reverberation have a detrimental effect on the
performance of automatic speech recognition (ASR) systems. Multi-condition
training of neural network-based acoustic models is used to deal with this
problem, but it requires many-folds data augmentation, resulting in increased
training time. In this paper, we propose utterance-level noise vectors for
noise-aware training of acoustic models in hybrid ASR. Our noise vectors are
obtained by combining the means of speech frames and silence frames in the
utterance, where the speech/silence labels may be obtained from a GMM-HMM model
trained for ASR alignments, such that no extra computation is required beyond
averaging of feature vectors. We show through experiments on AMI and Aurora-4
that this simple adaptation technique can result in 6-7% relative WER
improvement. We implement several embedding-based adaptation baselines proposed
in literature, and show that our method outperforms them on both the datasets.
Finally, we extend our method to the online ASR setting by using frame-level
maximum likelihood for the mean estimation.
