  One major task of spoken language understanding (SLU) in modern personal
assistants is to extract semantic concepts from an utterance, called slot
filling. Although existing slot filling models attempted to improve extracting
new concepts that are not seen in training data, the performance in practice is
still not satisfied. Recent research collected question and answer annotated
data to learn what is unknown and should be asked, yet not practically scalable
due to the heavy data collection effort. In this paper, we incorporate
softmax-based slot filling neural architectures to model the sequence
uncertainty without question supervision. We design a Dirichlet Prior RNN to
model high-order uncertainty by degenerating as softmax layer for RNN model
training. To further enhance the uncertainty modeling robustness, we propose a
novel multi-task training to calibrate the Dirichlet concentration parameters.
We collect unseen concepts to create two test datasets from SLU benchmark
datasets Snips and ATIS. On these two and another existing Concept Learning
benchmark datasets, we show that our approach significantly outperforms
state-of-the-art approaches by up to 8.18%. Our method is generic and can be
applied to any RNN or Transformer based slot filling models with a softmax
layer.
