  We investigate the scalable image classification problem with a large number
of categories. Hierarchical visual data structures are helpful for improving
the efficiency and performance of large-scale multi-class classification. We
propose a novel image classification method based on learning hierarchical
inter-class structures. Specifically, we first design a fast algorithm to
compute the similarity metric between categories, based on which a visual tree
is constructed by hierarchical spectral clustering. Using the learned visual
tree, a test sample label is efficiently predicted by searching for the best
path over the entire tree. The proposed method is extensively evaluated on the
ILSVRC2010 and Caltech 256 benchmark datasets. Experimental results show that
our method obtains significantly better category hierarchies than other
state-of-the-art visual tree-based methods and, therefore, much more accurate
classification.
