  Extreme multi-label text classification (XMTC) is an important problem in the
era of big data, for tagging a given text with the most relevant multiple
labels from an extremely large-scale label set. XMTC can be found in many
applications, such as item categorization, web page tagging, and news
annotation. Traditionally most methods used bag-of-words (BOW) as inputs,
ignoring word context as well as deep semantic information. Recent attempts to
overcome the problems of BOW by deep learning still suffer from 1) failing to
capture the important subtext for each label and 2) lack of scalability against
the huge number of labels. We propose a new label tree-based deep learning
model for XMTC, called AttentionXML, with two unique features: 1) a multi-label
attention mechanism with raw text as input, which allows to capture the most
relevant part of text to each label; and 2) a shallow and wide probabilistic
label tree (PLT), which allows to handle millions of labels, especially for
"tail labels". We empirically compared the performance of AttentionXML with
those of eight state-of-the-art methods over six benchmark datasets, including
Amazon-3M with around 3 million labels. AttentionXML outperformed all competing
methods under all experimental settings. Experimental results also show that
AttentionXML achieved the best performance against tail labels among label
tree-based methods. The code and datasets are available at
http://github.com/yourh/AttentionXML .
