  Deep neural networks have shown excellent performances in many real-world
applications. Unfortunately, they may show "Clever Hans"-like behavior --
making use of confounding factors within datasets -- to achieve high
performance. In this work, we introduce the novel learning setting of
"explanatory interactive learning" (XIL) and illustrate its benefits on a plant
phenotyping research task. XIL adds the scientist into the training loop such
that she interactively revises the original model via providing feedback on its
explanations. Our experimental results demonstrate that XIL can help avoiding
Clever Hans moments in machine learning and encourages (or discourages, if
appropriate) trust into the underlying model.
