  Connectionist Temporal Classification has recently attracted a lot of
interest as it offers an elegant approach to building acoustic models (AMs) for
speech recognition. The CTC loss function maps an input sequence of observable
feature vectors to an output sequence of symbols. Output symbols are
conditionally independent of each other under CTC loss, so a language model
(LM) can be incorporated conveniently during decoding, retaining the
traditional separation of acoustic and linguistic components in ASR. For fixed
vocabularies, Weighted Finite State Transducers provide a strong baseline for
efficient integration of CTC AMs with n-gram LMs. Character-based neural LMs
provide a straight forward solution for open vocabulary speech recognition and
all-neural models, and can be decoded with beam search. Finally,
sequence-to-sequence models can be used to translate a sequence of individual
sounds into a word string. We compare the performance of these three
approaches, and analyze their error patterns, which provides insightful
guidance for future research and development in this important area.
