  Large-scale multi-label text classification (LMTC) aims to associate a
document with its relevant labels from a large candidate set. Most existing
LMTC approaches rely on massive human-annotated training data, which are often
costly to obtain and suffer from a long-tailed label distribution (i.e., many
labels occur only a few times in the training set). In this paper, we study
LMTC under the zero-shot setting, which does not require any annotated
documents with labels and only relies on label surface names and descriptions.
To train a classifier that calculates the similarity score between a document
and a label, we propose a novel metadata-induced contrastive learning (MICoL)
method. Different from previous text-based contrastive learning techniques,
MICoL exploits document metadata (e.g., authors, venues, and references of
research papers), which are widely available on the Web, to derive similar
document-document pairs. Experimental results on two large-scale datasets show
that: (1) MICoL significantly outperforms strong zero-shot text classification
and contrastive learning baselines; (2) MICoL is on par with the
state-of-the-art supervised metadata-aware LMTC method trained on 10K-200K
labeled documents; and (3) MICoL tends to predict more infrequent labels than
supervised methods, thus alleviates the deteriorated performance on long-tailed
labels.
