  High-level (e.g., semantic) features encoded in the latter layers of
convolutional neural networks are extensively exploited for image
classification, leaving low-level (e.g., color) features in the early layers
underexplored. In this paper, we propose a novel Decision Propagation Module
(DPM) to make an intermediate decision that could act as category-coherent
guidance extracted from early layers, and then propagate it to the latter
layers. Therefore, by stacking a collection of DPMs into a classification
network, the generated Decision Propagation Network is explicitly formulated as
to progressively encode more discriminative features guided by the decision,
and then refine the decision based on the new generated features layer by
layer. Comprehensive results on four publicly available datasets validate DPM
could bring significant improvements for existing classification networks with
minimal additional computational cost and is superior to the state-of-the-art
methods.
