  We study how to generate captions that are not only accurate in describing an
image but also discriminative across different images. The problem is both
fundamental and interesting, as most machine-generated captions, despite
phenomenal research progresses in the past several years, are expressed in a
very monotonic and featureless format. While such captions are normally
accurate, they often lack important characteristics in human languages -
distinctiveness for each caption and diversity for different images. To address
this problem, we propose a novel conditional generative adversarial network for
generating diverse captions across images. Instead of estimating the quality of
a caption solely on one image, the proposed comparative adversarial learning
framework better assesses the quality of captions by comparing a set of
captions within the image-caption joint space. By contrasting with
human-written captions and image-mismatched captions, the caption generator
effectively exploits the inherent characteristics of human languages, and
generates more discriminative captions. We show that our proposed network is
capable of producing accurate and diverse captions across images.
