  Slot Filling (SF) is one of the sub-tasks of Spoken Language Understanding
(SLU) which aims to extract semantic constituents from a given natural language
utterance. It is formulated as a sequence labeling task. Recently, it has been
shown that contextual information is vital for this task. However, existing
models employ contextual information in a restricted manner, e.g., using
self-attention. Such methods fail to distinguish the effects of the context on
the word representation and the word label. To address this issue, in this
paper, we propose a novel method to incorporate the contextual information in
two different levels, i.e., representation level and task-specific (i.e.,
label) level. Our extensive experiments on three benchmark datasets on SF show
the effectiveness of our model leading to new state-of-the-art results on all
three benchmark datasets for the task of SF.
