  In the context of fine-grained visual categorization, the ability to
interpret models as human-understandable visual manuals is sometimes as
important as achieving high classification accuracy. In this paper, we propose
a novel Part-Stacked CNN architecture that explicitly explains the fine-grained
recognition process by modeling subtle differences from object parts. Based on
manually-labeled strong part annotations, the proposed architecture consists of
a fully convolutional network to locate multiple object parts and a two-stream
classification network that en- codes object-level and part-level cues
simultaneously. By adopting a set of sharing strategies between the computation
of multiple object parts, the proposed architecture is very efficient running
at 20 frames/sec during inference. Experimental results on the CUB-200-2011
dataset reveal the effectiveness of the proposed architecture, from both the
perspective of classification accuracy and model interpretability.
