  Collecting paired training data is difficult in practice, but the unpaired
samples broadly exist. Current approaches aim at generating synthesized
training data from unpaired samples by exploring the relationship between the
corrupted and clean data. This work proposes LUD-VAE, a deep generative method
to learn the joint probability density function from data sampled from marginal
distributions. Our approach is based on a carefully designed probabilistic
graphical model in which the clean and corrupted data domains are conditionally
independent. Using variational inference, we maximize the evidence lower bound
(ELBO) to estimate the joint probability density function. Furthermore, we show
that the ELBO is computable without paired samples under the inference
invariant assumption. This property provides the mathematical rationale of our
approach in the unpaired setting. Finally, we apply our method to real-world
image denoising, super-resolution, and low-light image enhancement tasks and
train the models using the synthetic data generated by the LUD-VAE.
Experimental results validate the advantages of our method over other
approaches.
