  Hierarchical attention networks have recently achieved remarkable performance
for document classification in a given language. However, when multilingual
document collections are considered, training such models separately for each
language entails linear parameter growth and lack of cross-language transfer.
Learning a single multilingual model with fewer parameters is therefore a
challenging but potentially beneficial objective. To this end, we propose
multilingual hierarchical attention networks for learning document structures,
with shared encoders and/or shared attention mechanisms across languages, using
multi-task learning and an aligned semantic space as input. We evaluate the
proposed models on multilingual document classification with disjoint label
sets, on a large dataset which we provide, with 600k news documents in 8
languages, and 5k labels. The multilingual models outperform monolingual ones
in low-resource as well as full-resource settings, and use fewer parameters,
thus confirming their computational efficiency and the utility of
cross-language transfer.
