  Variational Autoencoders (VAE) and their variants have been widely used in a
variety of applications, such as dialog generation, image generation and
disentangled representation learning. However, the existing VAE models have
some limitations in different applications. For example, a VAE easily suffers
from KL vanishing in language modeling and low reconstruction quality for
disentangling. To address these issues, we propose a novel controllable
variational autoencoder framework, ControlVAE, that combines a controller,
inspired by automatic control theory, with the basic VAE to improve the
performance of resulting generative models. Specifically, we design a new
non-linear PI controller, a variant of the proportional-integral-derivative
(PID) control, to automatically tune the hyperparameter (weight) added in the
VAE objective using the output KL-divergence as feedback during model training.
The framework is evaluated using three applications; namely, language modeling,
disentangled representation learning, and image generation. The results show
that ControlVAE can achieve better disentangling and reconstruction quality
than the existing methods. For language modelling, it not only averts the
KL-vanishing, but also improves the diversity of generated text. Finally, we
also demonstrate that ControlVAE improves the reconstruction quality of
generated images compared to the original VAE.
