  We consider independent component analysis of binary data. While fundamental
in practice, this case has been much less developed than ICA for continuous
data. We start by assuming a linear mixing model in a continuous-valued latent
space, followed by a binary observation model. Importantly, we assume that the
sources are non-stationary; this is necessary since any non-Gaussianity would
essentially be destroyed by the binarization. Interestingly, the model allows
for closed-form likelihood by employing the cumulative distribution function of
the multivariate Gaussian distribution. In stark contrast to the
continuous-valued case, we prove non-identifiability of the model with few
observed variables; our empirical results imply identifiability when the number
of observed variables is higher. We present a practical method for binary ICA
that uses only pairwise marginals, which are faster to compute than the full
multivariate likelihood. Experiments give insight into the requirements for the
number of observed variables, segments, and latent sources that allow the model
to be estimated.
