  This paper presents Hierarchical Network Dissection, a general pipeline to
interpret the internal representation of face-centric inference models. Using a
probabilistic formulation, our pipeline pairs units of the model with concepts
in our "Face Dictionary", a collection of facial concepts with corresponding
sample images. Our pipeline is inspired by Network Dissection, a popular
interpretability model for object-centric and scene-centric models. However,
our formulation allows to deal with two important challenges of face-centric
models that Network Dissection cannot address: (1) spacial overlap of concepts:
there are different facial concepts that simultaneously occur in the same
region of the image, like "nose" (facial part) and "pointy nose" (facial
attribute); and (2) global concepts: there are units with affinity to concepts
that do not refer to specific locations of the face (e.g. apparent age). We use
Hierarchical Network Dissection to dissect different face-centric inference
models trained on widely-used facial datasets. The results show models trained
for different tasks learned different internal representations. Furthermore,
the interpretability results can reveal some biases in the training data and
some interesting characteristics of the face-centric inference tasks. Finally,
we conduct controlled experiments on biased data to showcase the potential of
Hierarchical Network Dissection for bias discovery. The results illustrate how
Hierarchical Network Dissection can be used to discover and quantify bias in
the training data that is also encoded in the model.
