  Research in Image Generation has recently made significant progress,
particularly boosted by the introduction of Vision-Language models which are
able to produce high-quality visual content based on textual inputs. Despite
ongoing advancements in terms of generation quality and realism, no methodical
frameworks have been defined yet to quantitatively measure the quality of the
generated content and the adherence with the prompted requests: so far, only
human-based evaluations have been adopted for quality satisfaction and for
comparing different generative methods. We introduce a novel automated method
for Visual Concept Evaluation (ViCE), i.e. to assess consistency between a
generated/edited image and the corresponding prompt/instructions, with a
process inspired by the human cognitive behaviour. ViCE combines the strengths
of Large Language Models (LLMs) and Visual Question Answering (VQA) into a
unified pipeline, aiming to replicate the human cognitive process in quality
assessment. This method outlines visual concepts, formulates image-specific
verification questions, utilizes the Q&A system to investigate the image, and
scores the combined outcome. Although this brave new hypothesis of mimicking
humans in the image evaluation process is in its preliminary assessment stage,
results are promising and open the door to a new form of automatic evaluation
which could have significant impact as the image generation or the image target
editing tasks become more and more sophisticated.
