  The challenge that climate change poses to humanity has spurred a rapidly
developing field of artificial intelligence research focused on climate change
applications. The climate change AI (CCAI) community works on a diverse,
challenging set of problems which often involve physics-constrained ML or
heterogeneous spatiotemporal data. It would be desirable to use automated
machine learning (AutoML) techniques to automatically find high-performing
architectures and hyperparameters for a given dataset. In this work, we
benchmark popular AutoML libraries on three high-leverage CCAI applications:
climate modeling, wind power forecasting, and catalyst discovery. We find that
out-of-the-box AutoML libraries currently fail to meaningfully surpass the
performance of human-designed CCAI models. However, we also identify a few key
weaknesses, which stem from the fact that most AutoML techniques are tailored
to computer vision and NLP applications. For example, while dozens of search
spaces have been designed for image and language data, none have been designed
for spatiotemporal data. Addressing these key weaknesses can lead to the
discovery of novel architectures that yield substantial performance gains
across numerous CCAI applications. Therefore, we present a call to action to
the AutoML community, since there are a number of concrete, promising
directions for future work in the space of AutoML for CCAI. We release our code
and a list of resources at
https://github.com/climate-change-automl/climate-change-automl.
