  As the societal impact of Deep Neural Networks (DNNs) grows, the goals for
advancing DNNs become more complex and diverse, ranging from improving a
conventional model accuracy metric to infusing advanced human virtues such as
fairness, accountability, transparency (FaccT), and unbiasedness. Recently,
techniques in Explainable Artificial Intelligence (XAI) are attracting
considerable attention, and have tremendously helped Machine Learning (ML)
engineers in understanding AI models. However, at the same time, we started to
witness the emerging need beyond XAI among AI communities; based on the
insights learned from XAI, how can we better empower ML engineers in steering
their DNNs so that the model's reasonableness and performance can be improved
as intended? This article provides a timely and extensive literature overview
of the field Explanation-Guided Learning (EGL), a domain of techniques that
steer the DNNs' reasoning process by adding regularization, supervision, or
intervention on model explanations. In doing so, we first provide a formal
definition of EGL and its general learning paradigm. Secondly, an overview of
the key factors for EGL evaluation, as well as summarization and categorization
of existing evaluation procedures and metrics for EGL are provided. Finally,
the current and potential future application areas and directions of EGL are
discussed, and an extensive experimental study is presented aiming at providing
comprehensive comparative studies among existing EGL models in various popular
application domains, such as Computer Vision (CV) and Natural Language
Processing (NLP) domains.
