  Transformer-based pretrained language models (PLMs) offer unmatched
performance across the majority of natural language understanding (NLU) tasks,
including a body of question answering (QA) tasks. We hypothesize that
improvements in QA methodology can also be directly exploited in dialog NLU;
however, dialog tasks must be \textit{reformatted} into QA tasks. In
particular, we focus on modeling and studying \textit{slot labeling} (SL), a
crucial component of NLU for dialog, through the QA optics, aiming to improve
both its performance and efficiency, and make it more effective and resilient
to working with limited task data. To this end, we make a series of
contributions: 1) We demonstrate how QA-tuned PLMs can be applied to the SL
task, reaching new state-of-the-art performance, with large gains especially
pronounced in such low-data regimes. 2) We propose to leverage contextual
information, required to tackle ambiguous values, simply through natural
language. 3) Efficiency and compactness of QA-oriented fine-tuning are boosted
through the use of lightweight yet effective adapter modules. 4) Trading-off
some of the quality of QA datasets for their size, we experiment with larger
automatically generated QA datasets for QA-tuning, arriving at even higher
performance. Finally, our analysis suggests that our novel QA-based slot
labeling models, supported by the PLMs, reach a performance ceiling in
high-data regimes, calling for more challenging and more nuanced benchmarks in
future work.
