  In natural language processing, extreme multi-label text classification is an
emerging but essential task. The problem of extreme multi-label text
classification (XMTC) is to recall some of the most relevant labels for a text
from an extremely large label set. Large-scale pre-trained models have brought
a new trend to this problem. Though the large-scale pre-trained models have
made significant achievements on this problem, the valuable fine-tuned methods
have yet to be studied. Though label semantics have been introduced in XMTC,
the vast semantic gap between texts and labels has yet to gain enough
attention. This paper builds a new guide network (GUDN) to help fine-tune the
pre-trained model to instruct classification later. Furthermore, GUDN uses raw
label semantics combined with a helpful label reinforcement strategy to
effectively explore the latent space between texts and labels, narrowing the
semantic gap, which can further improve predicted accuracy. Experimental
results demonstrate that GUDN outperforms state-of-the-art methods on Eurlex-4k
and has competitive results on other popular datasets. In an additional
experiment, we investigated the input lengths' influence on the
Transformer-based model's accuracy. Our source code is released at
https://t.hk.uy/aFSH.
