  Natural Language Understanding (NLU) is an established component within a
conversational AI or digital assistant system, and it is responsible for
producing semantic understanding of a user request. We propose a scalable and
automatic approach for improving NLU in a large-scale conversational AI system
by leveraging implicit user feedback, with an insight that user interaction
data and dialog context have rich information embedded from which user
satisfaction and intention can be inferred. In particular, we propose a general
domain-agnostic framework for curating new supervision data for improving NLU
from live production traffic. With an extensive set of experiments, we show the
results of applying the framework and improving NLU for a large-scale
production system and show its impact across 10 domains.
