  Task-oriented dialogue systems in industry settings need to have high
conversational capability, be easily adaptable to changing situations and
conform to business constraints. This paper describes a 3-step procedure to
develop a conversational model that satisfies these criteria and can
efficiently scale to rank a large set of response candidates. First, we provide
a simple algorithm to semi-automatically create a high-coverage template set
from historic conversations without any annotation. Second, we propose a neural
architecture that encodes the dialogue context and applicable business
constraints as profile features for ranking the next turn. Third, we describe a
two-stage learning strategy with self-supervised training, followed by
supervised fine-tuning on limited data collected through a human-in-the-loop
platform. Finally, we describe offline experiments and present results of
deploying our model with human-in-the-loop to converse with live customers
online.
