  Visual storytelling consists in generating a natural language story given a
temporally ordered sequence of images. This task is not only challenging for
models, but also very difficult to evaluate with automatic metrics since there
is no consensus about what makes a story 'good'. In this paper, we introduce a
novel method that measures story quality in terms of human likeness regarding
three key aspects highlighted in previous work: visual grounding, coherence,
and repetitiveness. We then use this method to evaluate the stories generated
by several models, showing that the foundation model LLaVA obtains the best
result, but only slightly so compared to TAPM, a 50-times smaller visual
storytelling model. Upgrading the visual and language components of TAPM
results in a model that yields competitive performance with a relatively low
number of parameters. Finally, we carry out a human evaluation study, whose
results suggest that a 'good' story may require more than a human-like level of
visual grounding, coherence, and repetition.
