  Variational auto-encoders are powerful probabilistic models in generative
tasks but suffer from generating low-quality samples which are caused by the
holes in the prior. We propose the Coupled Variational Auto-Encoder (C-VAE),
which formulates the VAE problem as one of Optimal Transport (OT) between the
prior and data distributions. The C-VAE allows greater flexibility in priors
and natural resolution of the prior hole problem by enforcing coupling between
the prior and the data distribution and enables flexible optimization through
the primal, dual, and semi-dual formulations of entropic OT. Simulations on
synthetic and real data show that the C-VAE outperforms alternatives including
VAE, WAE, and InfoVAE in fidelity to the data, quality of the latent
representation, and in quality of generated samples.
