  Hierarchical text classification (HTC) is an important task with broad
applications, while few-shot HTC has gained increasing interest recently. While
in-context learning (ICL) with large language models (LLMs) has achieved
significant success in few-shot learning, it is not as effective for HTC
because of the expansive hierarchical label sets and extremely-ambiguous
labels. In this work, we introduce the first ICL-based framework with LLM for
few-shot HTC. We exploit a retrieval database to identify relevant
demonstrations, and an iterative policy to manage multi-layer hierarchical
labels. Particularly, we equip the retrieval database with HTC label-aware
representations for the input texts, which is achieved by continual training on
a pretrained language model with masked language modeling (MLM), layer-wise
classification (CLS, specifically for HTC), and a novel divergent contrastive
learning (DCL, mainly for adjacent semantically-similar labels) objective.
Experimental results on three benchmark datasets demonstrate superior
performance of our method, and we can achieve state-of-the-art results in
few-shot HTC.
