  Data-driven models are central to scientific discovery. In efforts to achieve
state-of-the-art model accuracy, researchers are employing increasingly complex
machine learning algorithms that often outperform simple regressions in
interpolative settings (e.g. random k-fold cross-validation) but suffer from
poor extrapolation performance, portability, and human interpretability, which
limits their potential for facilitating novel scientific insight. Here we
examine the trade-off between model performance and interpretability across a
broad range of science and engineering problems with an emphasis on materials
science datasets. We compare the performance of black box random forest and
neural network machine learning algorithms to that of single-feature linear
regressions which are fitted using interpretable input features discovered by a
simple random search algorithm. For interpolation problems, the average
prediction errors of linear regressions were twice as high as those of black
box models. Remarkably, when prediction tasks required extrapolation, linear
models yielded average error only 5% higher than that of black box models, and
outperformed black box models in roughly 40% of the tested prediction tasks,
which suggests that they may be desirable over complex algorithms in many
extrapolation problems because of their superior interpretability,
computational overhead, and ease of use. The results challenge the common
assumption that extrapolative models for scientific machine learning are
constrained by an inherent trade-off between performance and interpretability.
