  Large vision-language models (VLMs) have demonstrated remarkable abilities in
understanding everyday content. However, their performance in the domain of
art, particularly culturally rich art forms, remains less explored. As a pearl
of human wisdom and creativity, art encapsulates complex cultural narratives
and symbolism. In this paper, we offer the Pun Rebus Art Dataset, a multimodal
dataset for art understanding deeply rooted in traditional Chinese culture. We
focus on three primary tasks: identifying salient visual elements, matching
elements with their symbolic meanings, and explanations for the conveyed
messages. Our evaluation reveals that state-of-the-art VLMs struggle with these
tasks, often providing biased and hallucinated explanations and showing limited
improvement through in-context learning. By releasing the Pun Rebus Art
Dataset, we aim to facilitate the development of VLMs that can better
understand and interpret culturally specific content, promoting greater
inclusiveness beyond English-based corpora.
