  Recent algorithms in convolutional neural networks (CNN) considerably advance
the fine-grained image classification, which aims to differentiate subtle
differences among subordinate classes. However, previous studies have rarely
focused on learning a fined-grained and structured feature representation that
is able to locate similar images at different levels of relevance, e.g.,
discovering cars from the same make or the same model, both of which require
high precision. In this paper, we propose two main contributions to tackle this
problem. 1) A multi-task learning framework is designed to effectively learn
fine-grained feature representations by jointly optimizing both classification
and similarity constraints. 2) To model the multi-level relevance, label
structures such as hierarchy or shared attributes are seamlessly embedded into
the framework by generalizing the triplet loss. Extensive and thorough
experiments have been conducted on three fine-grained datasets, i.e., the
Stanford car, the car-333, and the food datasets, which contain either
hierarchical labels or shared attributes. Our proposed method has achieved very
competitive performance, i.e., among state-of-the-art classification accuracy.
More importantly, it significantly outperforms previous fine-grained feature
representations for image retrieval at different levels of relevance.
