  The ability to explain decisions made by machine learning models remains one
of the most significant hurdles towards widespread adoption of AI in highly
sensitive areas such as medicine, cybersecurity or autonomous driving. Great
interest exists in understanding which features of the input data prompt model
decision making. In this contribution, we propose a novel approach to identify
relevant features of the input data, inspired by methods from the energy
landscapes field, developed in the physical sciences. By identifying conserved
weights within groups of minima of the loss landscapes, we can identify the
drivers of model decision making. Analogues to this idea exist in the molecular
sciences, where coordinate invariants or order parameters are employed to
identify critical features of a molecule. However, no such approach exists for
machine learning loss landscapes. We will demonstrate the applicability of
energy landscape methods to machine learning models and give examples, both
synthetic and from the real world, for how these methods can help to make
models more interpretable.
