  Despite the promising performance of existing visual models on public
benchmarks, the critical assessment of their robustness for real-world
applications remains an ongoing challenge. To bridge this gap, we propose an
explainable visual dataset, XIMAGENET-12, to evaluate the robustness of visual
models. XIMAGENET-12 consists of over 200K images with 15,410 manual semantic
annotations. Specifically, we deliberately selected 12 categories from
ImageNet, representing objects commonly encountered in practical life. To
simulate real-world situations, we incorporated six diverse scenarios, such as
overexposure, blurring, and color changes, etc. We further develop a
quantitative criterion for robustness assessment, allowing for a nuanced
understanding of how visual models perform under varying conditions, notably in
relation to the background. We make the XIMAGENET-12 dataset and its
corresponding code openly accessible at
\url{https://sites.google.com/view/ximagenet-12/home}. We expect the
introduction of the XIMAGENET-12 dataset will empower researchers to thoroughly
evaluate the robustness of their visual models under challenging conditions.
