  Capturing the interesting components of an image is a key aspect of image
understanding. When a speaker annotates an image, selecting labels that are
informative greatly depends on the prior knowledge of a prospective listener.
Motivated by cognitive theories of categorization and communication, we present
a new unsupervised approach to model this prior knowledge and quantify the
informativeness of a description. Specifically, we compute how knowledge of a
label reduces uncertainty over the space of labels and utilize this to rank
candidate labels for describing an image. While the full estimation problem is
intractable, we describe an efficient algorithm to approximate entropy
reduction using a tree-structured graphical model. We evaluate our approach on
the open-images dataset using a new evaluation set of 10K ground-truth ratings
and find that it achieves ~65% agreement with human raters, largely
outperforming other unsupervised baseline approaches.
