  De novo design seeks to generate molecules with required property profiles by
virtual design-make-test cycles. With the emergence of deep learning and neural
generative models in many application areas, models for molecular design based
on neural networks appeared recently and show promising results. However, the
new models have not been profiled on consistent tasks, and comparative studies
to well-established algorithms have only seldom been performed.
  To standardize the assessment of both classical and neural models for de novo
molecular design, we propose an evaluation framework, GuacaMol, based on a
suite of standardized benchmarks. The benchmark tasks encompass measuring the
fidelity of the models to reproduce the property distribution of the training
sets, the ability to generate novel molecules, the exploration and exploitation
of chemical space, and a variety of single and multi-objective optimization
tasks. The benchmarking open-source Python code, and a leaderboard can be found
on https://benevolent.ai/guacamol
