  In conducting non-linear dimensionality reduction and feature learning, it is
common to suppose that the data lie near a lower-dimensional manifold. A class
of model-based approaches for such problems includes latent variables in an
unknown non-linear regression function; this includes Gaussian process latent
variable models and variational auto-encoders (VAEs) as special cases. VAEs are
artificial neural networks (ANNs) that employ approximations to make
computation tractable; however, current implementations lack adequate
uncertainty quantification in estimating the parameters, predictive densities,
and lower-dimensional subspace, and can be unstable and lack interpretability
in practice. We attempt to solve these problems by deploying Markov chain Monte
Carlo sampling algorithms (MCMC) for Bayesian inference in ANN models with
latent variables. We address issues of identifiability by imposing constraints
on the ANN parameters as well as by using anchor points. This is demonstrated
on simulated and real data examples. We find that current MCMC sampling schemes
face fundamental challenges in neural networks involving latent variables,
motivating new research directions.
