  To tackle increasingly complex tasks, it has become an essential ability of
neural networks to learn abstract representations. These task-specific
representations and, particularly, the invariances they capture turn neural
networks into black box models that lack interpretability. To open such a black
box, it is, therefore, crucial to uncover the different semantic concepts a
model has learned as well as those that it has learned to be invariant to. We
present an approach based on INNs that (i) recovers the task-specific, learned
invariances by disentangling the remaining factor of variation in the data and
that (ii) invertibly transforms these recovered invariances combined with the
model representation into an equally expressive one with accessible semantic
concepts. As a consequence, neural network representations become
understandable by providing the means to (i) expose their semantic meaning,
(ii) semantically modify a representation, and (iii) visualize individual
learned semantic concepts and invariances. Our invertible approach
significantly extends the abilities to understand black box models by enabling
post-hoc interpretations of state-of-the-art networks without compromising
their performance. Our implementation is available at
https://compvis.github.io/invariances/ .
