  Quantitatively assessing relationships between latent variables and observed
variables is important for understanding and developing generative models and
representation learning. In this paper, we propose latent-observed
dissimilarity (LOD) to evaluate the dissimilarity between the probabilistic
characteristics of latent and observed variables. We also define four essential
types of generative models with different independence/conditional independence
configurations. Experiments using tractable real-world data show that LOD can
effectively capture the differences between models and reflect the capability
for higher layer learning. They also show that the conditional independence of
latent variables given observed variables contributes to improving the
transmission of information and characteristics from lower layers to higher
layers.
