  Machine learning-based design has gained traction in the sciences, most
notably in the design of small molecules, materials, and proteins, with
societal implications spanning drug development and manufacturing, plastic
degradation, and carbon sequestration. When designing objects to achieve novel
property values with machine learning, one faces a fundamental challenge: how
to push past the frontier of current knowledge, distilled from the training
data into the model, in a manner that rationally controls the risk of failure.
If one trusts learned models too much in extrapolation, one is likely to design
rubbish. In contrast, if one does not extrapolate, one cannot find novelty.
Herein, we ponder how one might strike a useful balance between these two
extremes. We focus in particular on designing proteins with novel property
values, although much of our discussion addresses machine learning-based design
more broadly.
