  Current wisdom suggests more labeled image data is always better, and
obtaining labels is the bottleneck. Yet curating a pool of sufficiently diverse
and informative images is itself a challenge. In particular, training image
curation is problematic for fine-grained attributes, where the subtle visual
differences of interest may be rare within traditional image sources. We
propose an active image generation approach to address this issue. The main
idea is to jointly learn the attribute ranking task while also learning to
generate novel realistic image samples that will benefit that task. We
introduce an end-to-end framework that dynamically "imagines" image pairs that
would confuse the current model, presents them to human annotators for
labeling, then improves the predictive model with the new examples. With
results on two datasets, we show that by thinking outside the pool of real
images, our approach gains generalization accuracy for challenging fine-grained
attribute comparisons.
