  Automated audio captioning is a cross-modal translation task for describing
the content of audio clips with natural language sentences. This task has
attracted increasing attention and substantial progress has been made in recent
years. Captions generated by existing models are generally faithful to the
content of audio clips, however, these machine-generated captions are often
deterministic (e.g., generating a fixed caption for a given audio clip), simple
(e.g., using common words and simple grammar), and generic (e.g., generating
the same caption for similar audio clips). When people are asked to describe
the content of an audio clip, different people tend to focus on different sound
events and describe an audio clip diversely from various aspects using distinct
words and grammar. We believe that an audio captioning system should have the
ability to generate diverse captions, either for a fixed audio clip, or across
similar audio clips. To this end, we propose an adversarial training framework
based on a conditional generative adversarial network (C-GAN) to improve
diversity of audio captioning systems. A caption generator and two hybrid
discriminators compete and are learned jointly, where the caption generator can
be any standard encoder-decoder captioning model used to generate captions, and
the hybrid discriminators assess the generated captions from different
criteria, such as their naturalness and semantics. We conduct experiments on
the Clotho dataset. The results show that our proposed model can generate
captions with better diversity as compared to state-of-the-art methods.
