  In the last decade of automatic speech recognition (ASR) research, the
introduction of deep learning brought considerable reductions in word error
rate of more than 50% relative, compared to modeling without deep learning. In
the wake of this transition, a number of all-neural ASR architectures were
introduced. These so-called end-to-end (E2E) models provide highly integrated,
completely neural ASR models, which rely strongly on general machine learning
knowledge, learn more consistently from data, while depending less on ASR
domain-specific experience. The success and enthusiastic adoption of deep
learning accompanied by more generic model architectures lead to E2E models now
becoming the prominent ASR approach. The goal of this survey is to provide a
taxonomy of E2E ASR models and corresponding improvements, and to discuss their
properties and their relation to the classical hidden Markov model (HMM) based
ASR architecture. All relevant aspects of E2E ASR are covered in this work:
modeling, training, decoding, and external language model integration,
accompanied by discussions of performance and deployment opportunities, as well
as an outlook into potential future developments.
