  Deep generative models have been enjoying success in modeling continuous
data. However it remains challenging to capture the representations for
discrete structures with formal grammars and semantics, e.g., computer programs
and molecular structures. How to generate both syntactically and semantically
correct data still remains largely an open problem. Inspired by the theory of
compiler where the syntax and semantics check is done via syntax-directed
translation (SDT), we propose a novel syntax-directed variational autoencoder
(SD-VAE) by introducing stochastic lazy attributes. This approach converts the
offline SDT check into on-the-fly generated guidance for constraining the
decoder. Comparing to the state-of-the-art methods, our approach enforces
constraints on the output space so that the output will be not only
syntactically valid, but also semantically reasonable. We evaluate the proposed
model with applications in programming language and molecules, including
reconstruction and program/molecule optimization. The results demonstrate the
effectiveness in incorporating syntactic and semantic constraints in discrete
generative models, which is significantly better than current state-of-the-art
approaches.
