  Independent Component Analysis (ICA) is an effective unsupervised tool to
learn statistically independent representation. However, ICA is not only
sensitive to whitening but also difficult to learn an over-complete basis.
Consequently, ICA with soft Reconstruction cost(RICA) was presented to learn
sparse representations with over-complete basis even on unwhitened data.
Whereas RICA is infeasible to represent the data with nonlinear structure due
to its intrinsic linearity. In addition, RICA is essentially an unsupervised
method and can not utilize the class information. In this paper, we propose a
kernel ICA model with reconstruction constraint (kRICA) to capture the
nonlinear features. To bring in the class information, we further extend the
unsupervised kRICA to a supervised one by introducing a discrimination
constraint, namely d-kRICA. This constraint leads to learn a structured basis
consisted of basis vectors from different basis subsets corresponding to
different class labels. Then each subset will sparsely represent well for its
own class but not for the others. Furthermore, data samples belonging to the
same class will have similar representations, and thereby the learned sparse
representations can take more discriminative power. Experimental results
validate the effectiveness of kRICA and d-kRICA for image classification.
