  Response generation for task-oriented dialogues implicitly optimizes two
objectives at the same time: task completion and language quality. Conditioned
response generation serves as an effective approach to separately and better
optimize these two objectives. Such an approach relies on system action
annotations which are expensive to obtain. To alleviate the need of action
annotations, latent action learning is introduced to map each utterance to a
latent representation. However, this approach is prone to over-dependence on
the training data, and the generalization capability is thus restricted. To
address this issue, we propose to learn natural language actions that represent
utterances as a span of words. This explicit action representation promotes
generalization via the compositional structure of language. It also enables an
explainable generation process. Our proposed unsupervised approach learns a
memory component to summarize system utterances into a short span of words. To
further promote a compact action representation, we propose an auxiliary task
that restores state annotations as the summarized dialogue context using the
memory component. Our proposed approach outperforms latent action baselines on
MultiWOZ, a benchmark multi-domain dataset.
