  In consequential domains such as recidivism prediction, facility inspection,
and benefit assignment, it's important for individuals to know the
decision-relevant information for the model's prediction. In addition,
predictions should be fair both in terms of the outcome and the justification
of the outcome. In other words, decision-relevant features should provide
sufficient information for the predicted outcome and should be independent of
the membership of individuals in protected groups such as race and gender. In
this work, we focus on the problem of (un)fairness in the justification of the
text-based neural models. We tie the explanatory power of the model to fairness
in the outcome and propose a fairness-aware summarization mechanism to detect
and counteract the bias in such models. Given a potentially biased natural
language explanation for a decision, we use a multi-task neural model and an
attribution mechanism based on integrated gradients to extract high-utility and
low-bias justifications in form of a summary. The extracted summary is then
used for training a model to make decisions for individuals. Results on several
real world datasets suggest that our method drastically limits the demographic
leakage in the input (fairness in justification) while moderately enhancing the
fairness in the outcome. Our model is also effective in detecting and
counteracting several types of data poisoning attacks that synthesize
race-coded reasoning or irrelevant justifications.
