  Spoken language understanding (SLU) systems extract both text transcripts and
semantics associated with intents and slots from input speech utterances. SLU
systems usually consist of (1) an automatic speech recognition (ASR) module,
(2) an interface module that exposes relevant outputs from ASR, and (3) a
natural language understanding (NLU) module. Interfaces in SLU systems carry
information on text transcriptions or richer information like neural embeddings
from ASR to NLU. In this paper, we study how interfaces affect joint-training
for spoken language understanding. Most notably, we obtain the state-of-the-art
results on the publicly available 50-hr SLURP dataset. We first leverage
large-size pretrained ASR and NLU models that are connected by a text
interface, and then jointly train both models via a sequence loss function. For
scenarios where pretrained models are not utilized, the best results are
obtained through a joint sequence loss training using richer neural interfaces.
Finally, we show the overall diminishing impact of leveraging pretrained models
with increased training data size.
