  The goal of this paper is to discover a set of discriminative patches which
can serve as a fully unsupervised mid-level visual representation. The desired
patches need to satisfy two requirements: 1) to be representative, they need to
occur frequently enough in the visual world; 2) to be discriminative, they need
to be different enough from the rest of the visual world. The patches could
correspond to parts, objects, "visual phrases", etc. but are not restricted to
be any one of them. We pose this as an unsupervised discriminative clustering
problem on a huge dataset of image patches. We use an iterative procedure which
alternates between clustering and training discriminative classifiers, while
applying careful cross-validation at each step to prevent overfitting. The
paper experimentally demonstrates the effectiveness of discriminative patches
as an unsupervised mid-level visual representation, suggesting that it could be
used in place of visual words for many tasks. Furthermore, discriminative
patches can also be used in a supervised regime, such as scene classification,
where they demonstrate state-of-the-art performance on the MIT Indoor-67
dataset.
