  Artificial intelligence algorithms have been used to enhance a wide variety
of products and services, including assisting human decision making in
high-stakes contexts. However, these algorithms are complex and have
trade-offs, notably between prediction accuracy and fairness to population
subgroups. This makes it hard for designers to understand algorithms and design
products or services in a way that respects users' goals, values, and needs. We
proposed a method to help designers and users explore algorithms, visualize
their trade-offs, and select algorithms with trade-offs consistent with their
goals and needs. We evaluated our method on the problem of predicting criminal
defendants' likelihood to re-offend through (i) a large-scale Amazon Mechanical
Turk experiment, and (ii) in-depth interviews with domain experts. Our
evaluations show that our method can help designers and users of these systems
better understand and navigate algorithmic trade-offs. This paper contributes a
new way of providing designers the ability to understand and control the
outcomes of algorithmic systems they are creating.
