  Extreme multi-label text classification (XMC) seeks to find relevant labels
from an extreme large label collection for a given text input. Many real-world
applications can be formulated as XMC problems, such as recommendation systems,
document tagging and semantic search. Recently, transformer based XMC methods,
such as X-Transformer and LightXML, have shown significant improvement over
other XMC methods. Despite leveraging pre-trained transformer models for text
representation, the fine-tuning procedure of transformer models on large label
space still has lengthy computational time even with powerful GPUs. In this
paper, we propose a novel recursive approach, XR-Transformer to accelerate the
procedure through recursively fine-tuning transformer models on a series of
multi-resolution objectives related to the original XMC objective function.
Empirical results show that XR-Transformer takes significantly less training
time compared to other transformer-based XMC models while yielding better
state-of-the-art results. In particular, on the public Amazon-3M dataset with 3
million labels, XR-Transformer is not only 20x faster than X-Transformer but
also improves the Precision@1 from 51% to 54%.
