  Natural Language Generation (NLG) for task-oriented dialogue systems focuses
on communicating specific content accurately, fluently, and coherently. While
these attributes are crucial for a successful dialogue, it is also desirable to
simultaneously accomplish specific stylistic goals, such as response length,
point-of-view, descriptiveness, sentiment, formality, and empathy. In this
work, we focus on stylistic control and evaluation for schema-guided NLG, with
joint goals of achieving both semantic and stylistic control. We experiment in
detail with various controlled generation methods for large pretrained language
models: specifically, conditional training, guided fine-tuning, and guided
decoding. We discuss their advantages and limitations, and evaluate them with a
broad range of automatic and human evaluation metrics. Our results show that
while high style accuracy and semantic correctness are easier to achieve for
more lexically-defined styles with conditional training, stylistic control is
also achievable for more semantically complex styles using discriminator-based
guided decoding methods. The results also suggest that methods that are more
scalable (with less hyper-parameters tuning) and that disentangle content
generation and stylistic variations are more effective at achieving semantic
correctness and style accuracy.
