  In nonlinear latent variable models or dynamic models, if we consider the
latent variables as confounders (common causes), the noise dependencies imply
further relations between the observed variables. Such models are then closely
related to causal discovery in the presence of nonlinear confounders, which is
a challenging problem. However, generally in such models the observation noise
is assumed to be independent across data dimensions, and consequently the noise
dependencies are ignored. In this paper we focus on the Gaussian process latent
variable model (GPLVM), from which we develop an extended model called
invariant GPLVM (IGPLVM), which can adapt to arbitrary noise covariances. With
the Gaussian process prior put on a particular transformation of the latent
nonlinear functions, instead of the original ones, the algorithm for IGPLVM
involves almost the same computational loads as that for the original GPLVM.
Besides its potential application in causal discovery, IGPLVM has the advantage
that its estimated latent nonlinear manifold is invariant to any nonsingular
linear transformation of the data. Experimental results on both synthetic and
realworld data show its encouraging performance in nonlinear manifold learning
and causal discovery.
