  We present a syntax-infused variational autoencoder (SIVAE), that integrates
sentences with their syntactic trees to improve the grammar of generated
sentences. Distinct from existing VAE-based text generative models, SIVAE
contains two separate latent spaces, for sentences and syntactic trees. The
evidence lower bound objective is redesigned correspondingly, by optimizing a
joint distribution that accommodates two encoders and two decoders. SIVAE works
with long short-term memory architectures to simultaneously generate sentences
and syntactic trees. Two versions of SIVAE are proposed: one captures the
dependencies between the latent variables through a conditional prior network,
and the other treats the latent variables independently such that
syntactically-controlled sentence generation can be performed. Experimental
results demonstrate the generative superiority of SIVAE on both reconstruction
and targeted syntactic evaluations. Finally, we show that the proposed models
can be used for unsupervised paraphrasing given different syntactic tree
templates.
