  Popular conversational agents frameworks such as Alexa Skills Kit (ASK) and
Google Actions (gActions) offer unprecedented opportunities for facilitating
the development and deployment of voice-enabled AI solutions in various
verticals. Nevertheless, understanding user utterances with high accuracy
remains a challenging task with these frameworks. Particularly, when building
chatbots with large volume of domain-specific entities. In this paper, we
describe the challenges and lessons learned from building a large scale virtual
assistant for understanding and responding to equipment-related complaints. In
the process, we describe an alternative scalable framework for: 1) extracting
the knowledge about equipment components and their associated problem entities
from short texts, and 2) learning to identify such entities in user utterances.
We show through evaluation on a real dataset that the proposed framework,
compared to off-the-shelf popular ones, scales better with large volume of
entities being up to 30% more accurate, and is more effective in understanding
user utterances with domain-specific entities.
