  Intent detection (ID) and Slot filling (SF) are two major tasks in spoken
language understanding (SLU). Recently, attention mechanism has been shown to
be effective in jointly optimizing these two tasks in an interactive manner.
However, latest attention-based works concentrated only on the first-order
attention design, while ignoring the exploration of higher-order attention
mechanisms. In this paper, we propose a BiLinear attention block, which
leverages bilinear pooling to simultaneously exploit both the contextual and
channel-wise bilinear attention distributions to capture the second-order
interactions between the input intent or slot features. Higher and even
infinity order interactions are built by stacking numerous blocks and assigning
Exponential Linear Unit (ELU) to blocks. Before the decoding stage, we
introduce the Dynamic Feature Fusion Layer to implicitly fuse intent and slot
information in a more effective way. Technically, instead of simply
concatenating intent and slot features, we first compute two correlation
matrices to weight on two features. Furthermore, we present Higher-order
Attention Network for the SLU tasks. Experiments on two benchmark datasets show
that our approach yields improvements compared with the state-of-the-art
approach. We also provide discussion to demonstrate the effectiveness of the
proposed approach.
