  Many computer science disciplines (e.g., combinatorial optimization, natural
language processing, and information retrieval) use standard or established
test suites for evaluating algorithms. In visualization, similar approaches
have been adopted in some areas (e.g., volume visualization), while user
testimonies and empirical studies have been the dominant means of evaluation in
most other areas, such as designing colormaps. In this paper, we propose to
establish a test suite for evaluating the design of colormaps. With such a
suite, the users can observe the effects when different continuous colormaps
are applied to planar scalar fields that may exhibit various characteristic
features, such as jumps, local extrema, ridge or valley lines, different
distributions of scalar values, different gradients, different signal
frequencies, different levels of noise, and so on. The suite also includes an
expansible collection of real-world data sets including the most popular data
for colormap testing in the visualization literature. The test suite has been
integrated into a web-based application for creating continuous colormaps
(https://ccctool.com/), facilitating close inter-operation between design and
evaluation processes. This new facility complements traditional evaluation
methods such as user testimonies and empirical studies.
