  Recent work has raised concerns on the risk of unintended bias in AI systems
being used nowadays that can affect individuals unfairly based on race, gender
or religion, among other possible characteristics. While a lot of bias metrics
and fairness definitions have been proposed in recent years, there is no
consensus on which metric/definition should be used and there are very few
available resources to operationalize them. Therefore, despite recent
awareness, auditing for bias and fairness when developing and deploying AI
systems is not yet a standard practice. We present Aequitas, an open source
bias and fairness audit toolkit that is an intuitive and easy to use addition
to the machine learning workflow, enabling users to seamlessly test models for
several bias and fairness metrics in relation to multiple population
sub-groups. Aequitas facilitates informed and equitable decisions around
developing and deploying algorithmic decision making systems for both data
scientists, machine learning researchers and policymakers.
