  We introduce ImportantAug, a technique to augment training data for speech
classification and recognition models by adding noise to unimportant regions of
the speech and not to important regions. Importance is predicted for each
utterance by a data augmentation agent that is trained to maximize the amount
of noise it adds while minimizing its impact on recognition performance. The
effectiveness of our method is illustrated on version two of the Google Speech
Commands (GSC) dataset. On the standard GSC test set, it achieves a 23.3%
relative error rate reduction compared to conventional noise augmentation which
applies noise to speech without regard to where it might be most effective. It
also provides a 25.4% error rate reduction compared to a baseline without data
augmentation. Additionally, the proposed ImportantAug outperforms the
conventional noise augmentation and the baseline on two test sets with
additional noise added.
