  Today, many state-of-the-art automatic speech recognition (ASR) systems apply
all-neural models that map audio to word sequences trained end-to-end along one
global optimisation criterion in a fully data driven fashion. These models
allow high precision ASR for domains and words represented in the training
material but have difficulties recognising words that are rarely or not at all
represented during training, i.e. trending words and new named entities. In
this paper, we use a text-to-speech (TTS) engine to provide synthetic audio for
out-of-vocabulary (OOV) words. We aim to boost the recognition accuracy of a
recurrent neural network transducer (RNN-T) on OOV words by using the extra
audio-text pairs, while maintaining the performance on the non-OOV words.
Different regularisation techniques are explored and the best performance is
achieved by fine-tuning the RNN-T on both original training data and extra
synthetic data with elastic weight consolidation (EWC) applied on the encoder.
This yields a 57% relative word error rate (WER) reduction on utterances
containing OOV words without any degradation on the whole test set.
