  Counterfactual explanations are one of the most popular methods to make
predictions of black box machine learning models interpretable by providing
explanations in the form of `what-if scenarios'. Most current approaches
optimize a collapsed, weighted sum of multiple objectives, which are naturally
difficult to balance a-priori. We propose the Multi-Objective Counterfactuals
(MOC) method, which translates the counterfactual search into a multi-objective
optimization problem. Our approach not only returns a diverse set of
counterfactuals with different trade-offs between the proposed objectives, but
also maintains diversity in feature space. This enables a more detailed
post-hoc analysis to facilitate better understanding and also more options for
actionable user responses to change the predicted outcome. Our approach is also
model-agnostic and works for numerical and categorical input features. We show
the usefulness of MOC in concrete cases and compare our approach with
state-of-the-art methods for counterfactual explanations.
