  The remarkable success of transformers in the field of natural language
processing has sparked the interest of the speech-processing community, leading
to an exploration of their potential for modeling long-range dependencies
within speech sequences. Recently, transformers have gained prominence across
various speech-related domains, including automatic speech recognition, speech
synthesis, speech translation, speech para-linguistics, speech enhancement,
spoken dialogue systems, and numerous multimodal applications. In this paper,
we present a comprehensive survey that aims to bridge research studies from
diverse subfields within speech technology. By consolidating findings from
across the speech technology landscape, we provide a valuable resource for
researchers interested in harnessing the power of transformers to advance the
field. We identify the challenges encountered by transformers in speech
processing while also offering insights into potential solutions to address
these issues.
