  An image is often said to be worth a thousand words, and certain images can
tell rich and insightful stories. Can these stories be told via image
captioning? Images from folklore genres, such as mythology, folk dance,
cultural signs, and symbols, are vital to every culture. Our research compares
the performance of four popular vision-language models (GPT-4V, Gemini Pro
Vision, LLaVA, and OpenFlamingo) in identifying culturally specific information
in such images and creating accurate and culturally sensitive image captions.
We also propose a new evaluation metric, Cultural Awareness Score (CAS),
dedicated to measuring the degree of cultural awareness in image captions. We
provide a dataset MOSAIC-1.5k, labeled with ground truth for images containing
cultural background and context, as well as a labeled dataset with assigned
Cultural Awareness Scores that can be used with unseen data. Creating
culturally appropriate image captions is valuable for scientific research and
can be beneficial for many practical applications. We envision that our work
will promote a deeper integration of cultural sensitivity in AI applications
worldwide. By making the dataset and Cultural Awareness Score available to the
public, we aim to facilitate further research in this area, encouraging the
development of more culturally aware AI systems that respect and celebrate
global diversity.
