  Intelligent systems designed for play-based interactions should be
contextually aware of the users and their surroundings. Spoken Dialogue Systems
(SDS) are critical for these interactive agents to carry out effective
goal-oriented communication with users in real-time. For the real-world (i.e.,
in-the-wild) deployment of such conversational agents, improving the Natural
Language Understanding (NLU) module of the goal-oriented SDS pipeline is
crucial, especially with limited task-specific datasets. This study explores
the potential benefits of a recently proposed transformer-based multi-task NLU
architecture, mainly to perform Intent Recognition on small-size
domain-specific educational game datasets. The evaluation datasets were
collected from children practicing basic math concepts via play-based
interactions in game-based learning settings. We investigate the NLU
performances on the initial proof-of-concept game datasets versus the
real-world deployment datasets and observe anticipated performance drops
in-the-wild. We have shown that compared to the more straightforward baseline
approaches, Dual Intent and Entity Transformer (DIET) architecture is robust
enough to handle real-world data to a large extent for the Intent Recognition
task on these domain-specific in-the-wild game datasets.
