  Climate change has increased the intensity, frequency, and duration of
extreme weather events and natural disasters across the world. While the
increased data on natural disasters improves the scope of machine learning (ML)
in this field, progress is relatively slow. One bottleneck is the lack of
benchmark datasets that would allow ML researchers to quantify their progress
against a standard metric. The objective of this short paper is to explore the
state of benchmark datasets for ML tasks related to natural disasters,
categorizing them according to the disaster management cycle. We compile a list
of existing benchmark datasets introduced in the past five years. We propose a
web platform - NADBenchmarks - where researchers can search for benchmark
datasets for natural disasters, and we develop a preliminary version of such a
platform using our compiled list. This paper is intended to aid researchers in
finding benchmark datasets to train their ML models on, and provide general
directions for topics where they can contribute new benchmark datasets.
