  This study assesses the ability of Large Vision-Language Models (LVLMs) to
differentiate between AI-generated and human-generated images. It introduces a
new automated benchmark construction method for this evaluation. The experiment
compared common LVLMs with human participants using a mixed dataset of AI and
human-created images. Results showed that LVLMs could distinguish between the
image types to some extent but exhibited a rightward bias, and perform
significantly worse compared to humans. To build on these findings, we
developed an automated benchmark construction process using AI. This process
involved topic retrieval, narrative script generation, error embedding, and
image generation, creating a diverse set of text-image pairs with intentional
errors. We validated our method through constructing two caparable benchmarks.
This study highlights the strengths and weaknesses of LVLMs in real-world
understanding and advances benchmark construction techniques, providing a
scalable and automatic approach for AI model evaluation.
