  It is desirable to include more controllable attributes to enhance the
diversity of generated responses in open-domain dialogue systems. However,
existing methods can generate responses with only one controllable attribute or
lack a flexible way to generate them with multiple controllable attributes. In
this paper, we propose a Progressively trained Hierarchical Encoder-Decoder
(PHED) to tackle this task. More specifically, PHED deploys Conditional
Variational AutoEncoder (CVAE) on Transformer to include one aspect of
attributes at one stage. A vital characteristic of the CVAE is to separate the
latent variables at each stage into two types: a global variable capturing the
common semantic features and a specific variable absorbing the attribute
information at that stage. PHED then couples the CVAE latent variables with the
Transformer encoder and is trained by minimizing a newly derived ELBO and
controlled losses to produce the next stage's input and produce responses as
required. Finally, we conduct extensive evaluations to show that PHED
significantly outperforms the state-of-the-art neural generation models and
produces more diverse responses as expected.
