  Task-oriented dialogue (TOD) systems aim to achieve specific goals through
interactive dialogue. Such tasks usually involve following specific workflows,
i.e. executing a sequence of actions in a particular order. While prior work
has focused on supervised learning methods to condition on past actions, they
do not explicitly optimize for compliance to a desired workflow. In this paper,
we propose a novel framework based on reinforcement learning (RL) to generate
dialogue responses that are aligned with a given workflow. Our framework
consists of ComplianceScorer, a metric designed to evaluate how well a
generated response executes the specified action, combined with an RL
opimization process that utilizes an interactive sampling technique. We
evaluate our approach on two TOD datasets, Action-Based Conversations Dataset
(ABCD) (Chen et al., 2021a) and MultiWOZ 2.2 (Zang et al., 2020) on a range of
automated and human evaluation metrics. Our findings indicate that our RL-based
framework outperforms baselines and is effective at enerating responses that
both comply with the intended workflows while being expressed in a natural and
fluent manner.
