  Latent space Energy-Based Models (EBMs), also known as energy-based priors,
have drawn growing interests in generative modeling. Fueled by its flexibility
in the formulation and strong modeling power of the latent space, recent works
built upon it have made interesting attempts aiming at the interpretability of
text modeling. However, latent space EBMs also inherit some flaws from EBMs in
data space; the degenerate MCMC sampling quality in practice can lead to poor
generation quality and instability in training, especially on data with complex
latent structures. Inspired by the recent efforts that leverage diffusion
recovery likelihood learning as a cure for the sampling issue, we introduce a
novel symbiosis between the diffusion models and latent space EBMs in a
variational learning framework, coined as the latent diffusion energy-based
model. We develop a geometric clustering-based regularization jointly with the
information bottleneck to further improve the quality of the learned latent
space. Experiments on several challenging tasks demonstrate the superior
performance of our model on interpretable text modeling over strong
counterparts.
