  Evaluating and comparing text-to-image models is a challenging problem.
Significant advances in the field have recently been made, piquing interest of
various industrial sectors. As a consequence, a gold standard in the field
should cover a variety of tasks and application contexts. In this paper a novel
evaluation approach is experimented, on the basis of: (i) a curated data set,
made by high-quality royalty-free image-text pairs, divided into ten
categories; (ii) a quantitative metric, the CLIP-score, (iii) a human
evaluation task to distinguish, for a given text, the real and the generated
images. The proposed method has been applied to the most recent models, i.e.,
DALLE2, Latent Diffusion, Stable Diffusion, GLIDE and Craiyon. Early
experimental results show that the accuracy of the human judgement is fully
coherent with the CLIP-score. The dataset has been made available to the
public.
