  In this paper, we propose supervised dictionary learning (SDL) by
incorporating information on class labels into the learning of the dictionary.
To this end, we propose to learn the dictionary in a space where the dependency
between the signals and their corresponding labels is maximized. To maximize
this dependency, the recently introduced Hilbert Schmidt independence criterion
(HSIC) is used. One of the main advantages of this novel approach for SDL is
that it can be easily kernelized by incorporating a kernel, particularly a
data-derived kernel such as normalized compression distance, into the
formulation. The learned dictionary is compact and the proposed approach is
fast. We show that it outperforms other unsupervised and supervised dictionary
learning approaches in the literature, using real-world data.
