  We present a generative model for multitask conditional language generation.
Our guiding hypothesis is that a shared set of latent skills underlies many
disparate language generation tasks, and that explicitly modelling these skills
in a task embedding space can help with both positive transfer across tasks and
with efficient adaptation to new tasks. We instantiate this task embedding
space as a latent variable in a latent variable sequence-to-sequence model. We
evaluate this hypothesis by curating a series of monolingual text-to-text
language generation datasets - covering a broad range of tasks and domains -
and comparing the performance of models both in the multitask and few-shot
regimes. We show that our latent task variable model outperforms other
sequence-to-sequence baselines on average across tasks in the multitask
setting. In the few-shot learning setting on an unseen test dataset (i.e., a
new task), we demonstrate that model adaptation based on inference in the
latent task space is more robust than standard fine-tuning based parameter
adaptation and performs comparably in terms of overall performance. Finally, we
examine the latent task representations learnt by our model and show that they
cluster tasks in a natural way.
