  Hierarchical text classification (HTC) is a complex subtask under multi-label
text classification, characterized by a hierarchical label taxonomy and data
imbalance. The best-performing models aim to learn a static representation by
combining document and hierarchical label information. However, the relevance
of document sections can vary based on the hierarchy level, necessitating a
dynamic document representation. To address this, we propose HiGen, a
text-generation-based framework utilizing language models to encode dynamic
text representations. We introduce a level-guided loss function to capture the
relationship between text and label name semantics. Our approach incorporates a
task-specific pretraining strategy, adapting the language model to in-domain
knowledge and significantly enhancing performance for classes with limited
examples. Furthermore, we present a new and valuable dataset called ENZYME,
designed for HTC, which comprises articles from PubMed with the goal of
predicting Enzyme Commission (EC) numbers. Through extensive experiments on the
ENZYME dataset and the widely recognized WOS and NYT datasets, our methodology
demonstrates superior performance, surpassing existing approaches while
efficiently handling data and mitigating class imbalance. The data and code
will be released publicly.
