  In recent years, several methods have been proposed for explaining individual
predictions of deep learning models, yet there has been little study of how to
aggregate these predictions to explain how such models view classes as a whole
in text classification tasks. In this work, we propose a method for explaining
classes using deep learning models and the Integrated Gradients feature
attribution technique by aggregating explanations of individual examples in
text classification to general descriptions of the classes. We demonstrate the
approach on Web register (genre) classification using the XML-R model and the
Corpus of Online Registers of English (CORE), finding that the method
identifies plausible and discriminative keywords characterizing all but the
smallest class.
