  Supervised training of abstractive language generation models results in
learning conditional probabilities over language sequences based on the
supervised training signal. When the training signal contains a variety of
writing styles, such models may end up learning an 'average' style that is
directly influenced by the training data make-up and cannot be controlled by
the needs of an application. We describe a family of model architectures
capable of capturing both generic language characteristics via shared model
parameters, as well as particular style characteristics via private model
parameters. Such models are able to generate language according to a specific
learned style, while still taking advantage of their power to model generic
language phenomena. Furthermore, we describe an extension that uses a mixture
of output distributions from all learned styles to perform on-the fly style
adaptation based on the textual input alone. Experimentally, we find that the
proposed models consistently outperform models that encapsulate single-style or
average-style language generation capabilities.
