  In this paper, we propose a deep probabilistic multi-view model that is
composed of a linear multi-view layer based on probabilistic canonical
correlation analysis (CCA) description in the latent space together with deep
generative networks as observation models. The network is designed to decompose
the variations of all views into a shared latent representation and a set of
view-specific components where the shared latent representation is intended to
describe the common underlying sources of variation among the views. An
efficient variational inference procedure is developed that approximates the
posterior distributions of the latent probabilistic multi-view layer while
taking into account the solution of probabilistic CCA. A generalization to
models with arbitrary number of views is also proposed. The empirical studies
confirm that the proposed deep generative multi-view model can successfully
extend deep variational inference to multi-view learning while it efficiently
integrates the relationship between multiple views to alleviate the difficulty
of learning.
