  Building on recent advances in image generation, we present a fully
data-driven approach to rendering markup into images. The approach is based on
diffusion models, which parameterize the distribution of data using a sequence
of denoising operations on top of a Gaussian noise distribution. We view the
diffusion denoising process as a sequential decision making process, and show
that it exhibits compounding errors similar to exposure bias issues in
imitation learning problems. To mitigate these issues, we adapt the scheduled
sampling algorithm to diffusion training. We conduct experiments on four markup
datasets: mathematical formulas (LaTeX), table layouts (HTML), sheet music
(LilyPond), and molecular images (SMILES). These experiments each verify the
effectiveness of the diffusion process and the use of scheduled sampling to fix
generation issues. These results also show that the markup-to-image task
presents a useful controlled compositional setting for diagnosing and analyzing
generative image models.
