  Datasets of visualization play a crucial role in automating data-driven
visualization pipelines, serving as the foundation for supervised model
training and algorithm benchmarking. In this paper, we survey the literature on
visualization datasets and provide a comprehensive overview of existing
visualization datasets, including their data types, formats, supported tasks,
and openness. We propose a what-why-how model for visualization datasets,
considering the content of the dataset (what), the supported tasks (why), and
the dataset construction process (how). This model provides a clear
understanding of the diversity and complexity of visualization datasets.
Additionally, we highlight the challenges faced by existing visualization
datasets, including the lack of standardization in data types and formats and
the limited availability of large-scale datasets. To address these challenges,
we suggest future research directions.
