  Multi-label classification (MLC) studies the problem where each instance is
associated with multiple relevant labels, which leads to the exponential growth
of output space. MLC encourages a popular framework named label compression
(LC) for capturing label dependency with dimension reduction. Nevertheless,
most existing LC methods failed to consider the influence of the feature space
or misguided by original problematic features, so that may result in
performance degeneration. In this paper, we present a compact learning (CL)
framework to embed the features and labels simultaneously and with mutual
guidance. The proposal is a versatile concept, hence the embedding way is
arbitrary and independent of the subsequent learning process. Following its
spirit, a simple yet effective implementation called compact multi-label
learning (CMLL) is proposed to learn a compact low-dimensional representation
for both spaces. CMLL maximizes the dependence between the embedded spaces of
the labels and features, and minimizes the loss of label space recovery
concurrently. Theoretically, we provide a general analysis for different
embedding methods. Practically, we conduct extensive experiments to validate
the effectiveness of the proposed method.
