  Implementing systems based on Machine Learning to detect fraud and other
Non-Technical Losses (NTL) is challenging: the data available is biased, and
the algorithms currently used are black-boxes that cannot be either easily
trusted or understood by stakeholders. This work explains our human-in-the-loop
approach to mitigate these problems in a real system that uses a supervised
model to detect Non-Technical Losses (NTL) for an international utility company
from Spain. This approach exploits human knowledge (e.g. from the data
scientists or the company's stakeholders) and the information provided by
explanatory methods to guide the system during the training process. This
simple, efficient method that can be easily implemented in other industrial
projects is tested in a real dataset and the results show that the derived
prediction model is better in terms of accuracy, interpretability, robustness
and flexibility.
