  Neural text generation models are likely to suffer from the low-diversity
problem. Various decoding strategies and training-based methods have been
proposed to promote diversity only by exploiting contextual features, but
rarely do they consider incorporating syntactic structure clues. In this work,
we propose using linguistic annotation, i.e., part-of-speech (POS), to guide
the text generation. In detail, we introduce POS Guided Softmax to explicitly
model two posterior probabilities: (i) next-POS, and (ii) next-token from the
vocabulary of the target POS. A POS Guided Sampling strategy is further
proposed to address the low-diversity problem by enriching the diversity of
POS. Extensive experiments and human evaluations show that, compared with
existing state-of-the-art methods, our POS Guided Softmax and Sampling (POSG)
can generate more diverse text while maintaining comparable quality.
