  Automatically evaluating the quality of image captions can be very
challenging since human language is quite flexible that there can be various
expressions for the same meaning. Most of the current captioning metrics rely
on token level matching between candidate caption and the ground truth label
sentences. It usually neglects the sentence-level information. Motivated by the
auto-encoder mechanism and contrastive representation learning advances, we
propose a learning-based metric for image captioning, which we call Intrinsic
Image Captioning Evaluation($I^2CE$). We develop three progressive model
structures to learn the sentence level representations--single branch model,
dual branches model, and triple branches model. Our empirical tests show that
$I^2CE$ trained with dual branches structure achieves better consistency with
human judgments to contemporary image captioning evaluation metrics.
Furthermore, We select several state-of-the-art image captioning models and
test their performances on the MS COCO dataset concerning both contemporary
metrics and the proposed $I^2CE$. Experiment results show that our proposed
method can align well with the scores generated from other contemporary
metrics. On this concern, the proposed metric could serve as a novel indicator
of the intrinsic information between captions, which may be complementary to
the existing ones.
