  Recently, neural networks have been successfully employed to improve upon
state-of-the-art performance in ad-hoc retrieval tasks via machine-learned
ranking functions. While neural retrieval models grow in complexity and impact,
little is understood about their correspondence with well-studied IR
principles. Recent work on interpretability in machine learning has provided
tools and techniques to understand neural models in general, yet there has been
little progress towards explaining ranking models.
  We investigate whether one can explain the behavior of neural ranking models
in terms of their congruence with well understood principles of document
ranking by using established theories from axiomatic IR. Axiomatic analysis of
information retrieval models has formalized a set of constraints on ranking
decisions that reasonable retrieval models should fulfill. We operationalize
this axiomatic thinking to reproduce rankings based on combinations of
elementary constraints. This allows us to investigate to what extent the
ranking decisions of neural rankers can be explained in terms of retrieval
axioms, and which axioms apply in which situations. Our experimental study
considers a comprehensive set of axioms over several representative neural
rankers. While the existing axioms can already explain the particularly
confident ranking decisions rather well, future work should extend the axiom
set to also cover the other still "unexplainable" neural IR rank decisions.
