  This paper proposes a novel approach to train deep neural networks by
unlocking the layer-wise dependency of backpropagation training. The approach
employs additional modules called local critic networks besides the main
network model to be trained, which are used to obtain error gradients without
complete feedforward and backward propagation processes. We propose a cascaded
learning strategy for these local networks. In addition, the approach is also
useful from multi-model perspectives, including structural optimization of
neural networks, computationally efficient progressive inference, and ensemble
classification for performance improvement. Experimental results show the
effectiveness of the proposed approach and suggest guidelines for determining
appropriate algorithm parameters.
