  This paper shows that CIDEr-D, a traditional evaluation metric for image
description, does not work properly on datasets where the number of words in
the sentence is significantly greater than those in the MS COCO Captions
dataset. We also show that CIDEr-D has performance hampered by the lack of
multiple reference sentences and high variance of sentence length. To bypass
this problem, we introduce CIDEr-R, which improves CIDEr-D, making it more
flexible in dealing with datasets with high sentence length variance. We
demonstrate that CIDEr-R is more accurate and closer to human judgment than
CIDEr-D; CIDEr-R is more robust regarding the number of available references.
Our results reveal that using Self-Critical Sequence Training to optimize
CIDEr-R generates descriptive captions. In contrast, when CIDEr-D is optimized,
the generated captions' length tends to be similar to the reference length.
However, the models also repeat several times the same word to increase the
sentence length.
