  This paper presents the use of non-autoregressive (NAR) approaches for joint
automatic speech recognition (ASR) and spoken language understanding (SLU)
tasks. The proposed NAR systems employ a Conformer encoder that applies
connectionist temporal classification (CTC) to transcribe the speech utterance
into raw ASR hypotheses, which are further refined with a bidirectional encoder
representations from Transformers (BERT)-like decoder. In the meantime, the
intent and slot labels of the utterance are predicted simultaneously using the
same decoder. Both Mask-CTC and self-conditioned CTC (SC-CTC) approaches are
explored for this study. Experiments conducted on the SLURP dataset show that
the proposed SC-Mask-CTC NAR system achieves 3.7% and 3.2% absolute gains in
SLU metrics and a competitive level of ASR accuracy, when compared to a
Conformer-Transformer based autoregressive (AR) model. Additionally, the NAR
systems achieve 6x faster decoding speed than the AR baseline.
