  Machine Learning (ML) and its applications have been transforming our lives
but it is also creating issues related to the development of fair, accountable,
transparent, and ethical Artificial Intelligence. As the ML models are not
fully comprehensible yet, it is obvious that we still need humans to be part of
algorithmic decision-making processes. In this paper, we consider a ML
framework that may accelerate model learning and improve its interpretability
by incorporating human experts into the model learning loop. We propose a novel
human-in-the-loop ML framework aimed at dealing with learning problems that the
cost of data annotation is high and the lack of appropriate data to model the
association between the target tasks and the input features. With an
application to precision dosing, our experimental results show that the
approach can learn interpretable rules from data and may potentially lower
experts' workload by replacing data annotation with rule representation
editing. The approach may also help remove algorithmic bias by introducing
experts' feedback into the iterative model learning process.
