  Image captioning, a popular topic in computer vision, has achieved
substantial progress in recent years. However, the distinctiveness of natural
descriptions is often overlooked in previous work. It is closely related to the
quality of captions, as distinctive captions are more likely to describe images
with their unique aspects. In this work, we propose a new learning method,
Contrastive Learning (CL), for image captioning. Specifically, via two
constraints formulated on top of a reference model, the proposed method can
encourage distinctiveness, while maintaining the overall quality of the
generated captions. We tested our method on two challenging datasets, where it
improves the baseline model by significant margins. We also showed in our
studies that the proposed method is generic and can be used for models with
various structures.
