  Independent component analysis (ICA) is a statistical method for transforming
an observable multidimensional random vector into components that are as
statistically independent as possible from each other.Usually the ICA framework
assumes a model according to which the observations are generated (such as a
linear transformation with additive noise). ICA over finite fields is a special
case of ICA in which both the observations and the independent components are
over a finite alphabet. In this work we consider a generalization of this
framework in which an observation vector is decomposed to its independent
components (as much as possible) with no prior assumption on the way it was
generated. This generalization is also known as Barlow's minimal redundancy
representation problem and is considered an open problem. We propose several
theorems and show that this NP hard problem can be accurately solved with a
branch and bound search tree algorithm, or tightly approximated with a series
of linear problems. Our contribution provides the first efficient and
constructive set of solutions to Barlow's problem.The minimal redundancy
representation (also known as factorial code) has many applications, mainly in
the fields of Neural Networks and Deep Learning. The Binary ICA (BICA) is also
shown to have applications in several domains including medical diagnosis,
multi-cluster assignment, network tomography and internet resource management.
In this work we show this formulation further applies to multiple disciplines
in source coding such as predictive coding, distributed source coding and
coding of large alphabet sources.
