  Explainability for machine learning models has gained considerable attention
within the research community given the importance of deploying more reliable
machine-learning systems. In computer vision applications, generative
counterfactual methods indicate how to perturb a model's input to change its
prediction, providing details about the model's decision-making. Current
methods tend to generate trivial counterfactuals about a model's decisions, as
they often suggest to exaggerate or remove the presence of the attribute being
classified. For the machine learning practitioner, these types of
counterfactuals offer little value, since they provide no new information about
undesired model or data biases. In this work, we identify the problem of
trivial counterfactual generation and we propose DiVE to alleviate it. DiVE
learns a perturbation in a disentangled latent space that is constrained using
a diversity-enforcing loss to uncover multiple valuable explanations about the
model's prediction. Further, we introduce a mechanism to prevent the model from
producing trivial explanations. Experiments on CelebA and Synbols demonstrate
that our model improves the success rate of producing high-quality valuable
explanations when compared to previous state-of-the-art methods. Code is
available at https://github.com/ElementAI/beyond-trivial-explanations.
