  The adoption of machine learning in high-stakes applications such as
healthcare and law has lagged in part because predictions are not accompanied
by explanations comprehensible to the domain user, who often holds the ultimate
responsibility for decisions and outcomes. In this paper, we propose an
approach to generate such explanations in which training data is augmented to
include, in addition to features and labels, explanations elicited from domain
users. A joint model is then learned to produce both labels and explanations
from the input features. This simple idea ensures that explanations are
tailored to the complexity expectations and domain knowledge of the consumer.
Evaluation spans multiple modeling techniques on a game dataset, a (visual)
aesthetics dataset, a chemical odor dataset and a Melanoma dataset showing that
our approach is generalizable across domains and algorithms. Results
demonstrate that meaningful explanations can be reliably taught to machine
learning algorithms, and in some cases, also improve modeling accuracy.
