  A key challenge in developing and deploying Machine Learning (ML) systems is
understanding their performance across a wide range of inputs. To address this
challenge, we created the What-If Tool, an open-source application that allows
practitioners to probe, visualize, and analyze ML systems, with minimal coding.
The What-If Tool lets practitioners test performance in hypothetical
situations, analyze the importance of different data features, and visualize
model behavior across multiple models and subsets of input data. It also lets
practitioners measure systems according to multiple ML fairness metrics. We
describe the design of the tool, and report on real-life usage at different
organizations.
