  A well-designed fine-grained categorization system usually has three
contradictory requirements: accuracy (the ability to identify objects among
subordinate categories); interpretability (the ability to provide
human-understandable explanation of recognition system behavior); and
efficiency (the speed of the system). To handle the trade-off between accuracy
and interpretability, we propose a novel "Deeper Part-Stacked CNN" architecture
armed with interpretability by modeling subtle differences between object
parts. The proposed architecture consists of a part localization network, a
two-stream classification network that simultaneously encodes object-level and
part-level cues, and a feature vectors fusion component. Specifically, the part
localization network is implemented by exploring a new paradigm for key point
localization that first samples a small number of representable pixels and then
determine their labels via a convolutional layer followed by a softmax layer.
We also use a cropping layer to extract part features and propose a scale
mean-max layer for feature fusion learning. Experimentally, our proposed method
outperform state-of-the-art approaches both in part localization task and
classification task on Caltech-UCSD Birds-200-2011. Moreover, by adopting a set
of sharing strategies between the computation of multiple object parts, our
single model is fairly efficient running at 32 frames/sec.
