  We propose a latent space energy-based prior model for text generation and
classification. The model stands on a generator network that generates the text
sequence based on a continuous latent vector. The energy term of the prior
model couples a continuous latent vector and a symbolic one-hot vector, so that
discrete category can be inferred from the observed example based on the
continuous latent vector. Such a latent space coupling naturally enables
incorporation of information bottleneck regularization to encourage the
continuous latent vector to extract information from the observed example that
is informative of the underlying category. In our learning method, the
symbol-vector coupling, the generator network and the inference network are
learned jointly. Our model can be learned in an unsupervised setting where no
category labels are provided. It can also be learned in semi-supervised setting
where category labels are provided for a subset of training examples. Our
experiments demonstrate that the proposed model learns well-structured and
meaningful latent space, which (1) guides the generator to generate text with
high quality, diversity, and interpretability, and (2) effectively classifies
text.
