  Fingerprint-based models for protein-ligand binding have demonstrated
outstanding success on benchmark datasets; however, these models may not learn
the correct binding rules. To assess this concern, we use in silico datasets
with known binding rules to develop a general framework for evaluating model
attribution. This framework identifies fragments that a model considers
necessary to achieve a particular score, sidestepping the need for a model to
be differentiable. Our results confirm that high-performing models may not
learn the correct binding rule, and suggest concrete steps that can remedy this
situation. We show that adding fragment-matched inactive molecules (decoys) to
the data reduces attribution false negatives, while attribution false positives
largely arise from the background correlation structure of molecular data.
Normalizing for these background correlations helps to reveal the true binding
logic. Our work highlights the danger of trusting attributions from
high-performing models and suggests that a closer examination of fingerprint
correlation structure and better decoy selection may help reduce
misattributions.
