  Systems aiming to aid consumers in their decision-making (e.g., by
implementing persuasive techniques) are more likely to be effective when
consumers trust them. However, recent research has demonstrated that the
machine learning algorithms that often underlie such technology can act
unfairly towards specific groups (e.g., by making more favorable predictions
for men than for women). An undesired disparate impact resulting from this kind
of algorithmic unfairness could diminish consumer trust and thereby undermine
the purpose of the system. We studied this effect by conducting a
between-subjects user study investigating how (gender-related) disparate impact
affected consumer trust in an app designed to improve consumers' financial
decision-making. Our results show that disparate impact decreased consumers'
trust in the system and made them less likely to use it. Moreover, we find that
trust was affected to the same degree across consumer groups (i.e., advantaged
and disadvantaged users) despite both of these consumer groups recognizing
their respective levels of personal benefit. Our findings highlight the
importance of fairness in consumer-oriented artificial intelligence systems.
