  Binding affinity optimization is crucial in early-stage drug discovery. While
numerous machine learning methods exist for predicting ligand potency, their
comparative efficacy remains unclear. This study evaluates the performance of
classical tree-based models and advanced neural networks in protein-ligand
binding affinity prediction. Our comprehensive benchmarking encompasses 2D
models utilizing ligand-only RDKit embeddings and Large Language Model (LLM)
ligand representations, as well as 3D neural networks incorporating bound
protein-ligand conformations. We assess these models across multiple standard
datasets, examining various predictive scenarios including classification,
ranking, regression, and active learning. Results indicate that simpler models
can surpass more complex ones in specific tasks, while 3D models leveraging
structural information become increasingly competitive with larger training
datasets containing compounds with labelled affinity data against multiple
targets. Pre-trained 3D models, by incorporating protein pocket environments,
demonstrate significant advantages in data-scarce scenarios for specific
binding pockets. Additionally, LLM pretraining on 2D ligand data enhances
complex model performance, providing versatile embeddings that outperform
traditional RDKit features in computational efficiency. Finally, we show that
combining 2D and 3D model strengths improves active learning outcomes beyond
current state-of-the-art approaches. These findings offer valuable insights for
optimizing machine learning strategies in drug discovery pipelines.
