  AI alignment refers to models acting towards human-intended goals,
preferences, or ethical principles. Given that most large-scale deep learning
models act as black boxes and cannot be manually controlled, analyzing the
similarity between models and humans can be a proxy measure for ensuring AI
safety. In this paper, we focus on the models' visual perception alignment with
humans, further referred to as AI-human visual alignment. Specifically, we
propose a new dataset for measuring AI-human visual alignment in terms of image
classification, a fundamental task in machine perception. In order to evaluate
AI-human visual alignment, a dataset should encompass samples with various
scenarios that may arise in the real world and have gold human perception
labels. Our dataset consists of three groups of samples, namely Must-Act (i.e.,
Must-Classify), Must-Abstain, and Uncertain, based on the quantity and clarity
of visual information in an image and further divided into eight categories.
All samples have a gold human perception label; even Uncertain (severely
blurry) sample labels were obtained via crowd-sourcing. The validity of our
dataset is verified by sampling theory, statistical theories related to survey
design, and experts in the related fields. Using our dataset, we analyze the
visual alignment and reliability of five popular visual perception models and
seven abstention methods. Our code and data is available at
https://github.com/jiyounglee-0523/VisAlign.
