  Large-Scale Multi-Label Text Classification (LMTC) includes tasks with
hierarchical label spaces, such as automatic assignment of ICD-9 codes to
discharge summaries. Performance of models in prior art is evaluated with
standard precision, recall, and F1 measures without regard for the rich
hierarchical structure. In this work we argue for hierarchical evaluation of
the predictions of neural LMTC models. With the example of the ICD-9 ontology
we describe a structural issue in the representation of the structured label
space in prior art, and propose an alternative representation based on the
depth of the ontology. We propose a set of metrics for hierarchical evaluation
using the depth-based representation. We compare the evaluation scores from the
proposed metrics with previously used metrics on prior art LMTC models for
ICD-9 coding in MIMIC-III. We also propose further avenues of research
involving the proposed ontological representation.
