  End-to-end spoken language understanding (SLU) has recently attracted
increasing interest. Compared to the conventional tandem-based approach that
combines speech recognition and language understanding as separate modules, the
new approach extracts users' intentions directly from the speech signals,
resulting in joint optimization and low latency. Such an approach, however, is
typically designed to process one intention at a time, which leads users to
take multiple rounds to fulfill their requirements while interacting with a
dialogue system. In this paper, we propose a streaming end-to-end framework
that can process multiple intentions in an online and incremental way. The
backbone of our framework is a unidirectional RNN trained with the
connectionist temporal classification (CTC) criterion. By this design, an
intention can be identified when sufficient evidence has been accumulated, and
multiple intentions can be identified sequentially. We evaluate our solution on
the Fluent Speech Commands (FSC) dataset and the intent detection accuracy is
about 97 % on all multi-intent settings. This result is comparable to the
performance of the state-of-the-art non-streaming models, but is achieved in an
online and incremental way. We also employ our model to a keyword spotting task
using the Google Speech Commands dataset and the results are also highly
promising.
