  Biocatalysis is a promising approach to sustainably synthesize
pharmaceuticals, complex natural products, and commodity chemicals at scale.
However, the adoption of biocatalysis is limited by our ability to select
enzymes that will catalyze their natural chemical transformation on non-natural
substrates. While machine learning and in silico directed evolution are
well-posed for this predictive modeling challenge, efforts to date have
primarily aimed to increase activity against a single known substrate, rather
than to identify enzymes capable of acting on new substrates of interest. To
address this need, we curate 6 different high-quality enzyme family screens
from the literature that each measure multiple enzymes against multiple
substrates. We compare machine learning-based compound-protein interaction
(CPI) modeling approaches from the literature used for predicting drug-target
interactions. Surprisingly, comparing these interaction-based models against
collections of independent (single task) enzyme-only or substrate-only models
reveals that current CPI approaches are incapable of learning interactions
between compounds and proteins in the current family level data regime. We
further validate this observation by demonstrating that our no-interaction
baseline can outperform CPI-based models from the literature used to guide the
discovery of kinase inhibitors. Given the high performance of non-interaction
based models, we introduce a new structure-based strategy for pooling residue
representations across a protein sequence. Altogether, this work motivates a
principled path forward in order to build and evaluate meaningful predictive
models for biocatalysis and other drug discovery applications.
