  Controllable image captioning models generate human-like image descriptions,
enabling some kind of control over the generated captions. This paper focuses
on controlling the caption length, i.e. a short and concise description or a
long and detailed one. Since existing image captioning datasets contain mostly
short captions, generating long captions is challenging. To address the
shortage of long training examples, we propose to enrich the dataset with
varying-length self-generated captions. These, however, might be of varying
quality and are thus unsuitable for conventional training. We introduce a novel
training strategy that selects the data points to be used at different times
during the training. Our method dramatically improves the length-control
abilities, while exhibiting SoTA performance in terms of caption quality. Our
approach is general and is shown to be applicable also to paragraph generation.
