  The problem of revealing botnet activity through Domain Generation Algorithm
(DGA) detection seems to be solved, considering that available deep learning
classifiers achieve accuracies of over 99.9%. However, these classifiers
provide a false sense of security as they are heavily biased and allow for
trivial detection bypass. In this work, we leverage explainable artificial
intelligence (XAI) methods to analyze the reasoning of deep learning
classifiers and to systematically reveal such biases. We show that eliminating
these biases from DGA classifiers considerably deteriorates their performance.
Nevertheless we are able to design a context-aware detection system that is
free of the identified biases and maintains the detection rate of state-of-the
art deep learning classifiers. In this context, we propose a visual analysis
system that helps to better understand a classifier's reasoning, thereby
increasing trust in and transparency of detection methods and facilitating
decision-making.
