  Recent advances in nonlinear Independent Component Analysis (ICA) provide a
principled framework for unsupervised feature learning and disentanglement. The
central idea in such works is that the latent components are assumed to be
independent conditional on some observed auxiliary variables, such as the
time-segment index. This requires manual segmentation of data into
non-stationary segments which is computationally expensive, inaccurate and
often impossible. These models are thus not fully unsupervised. We remedy these
limitations by combining nonlinear ICA with a Hidden Markov Model, resulting in
a model where a latent state acts in place of the observed segment-index. We
prove identifiability of the proposed model for a general mixing nonlinearity,
such as a neural network. We also show how maximum likelihood estimation of the
model can be done using the expectation-maximization algorithm. Thus, we
achieve a new nonlinear ICA framework which is unsupervised, more efficient, as
well as able to model underlying temporal dynamics.
