  This paper aims to accelerate the test-time computation of deep convolutional
neural networks (CNNs). Unlike existing methods that are designed for
approximating linear filters or linear responses, our method takes the
nonlinear units into account. We minimize the reconstruction error of the
nonlinear responses, subject to a low-rank constraint which helps to reduce the
complexity of filters. We develop an effective solution to this constrained
nonlinear optimization problem. An algorithm is also presented for reducing the
accumulated error when multiple layers are approximated. A whole-model speedup
ratio of 4x is demonstrated on a large network trained for ImageNet, while the
top-5 error rate is only increased by 0.9%. Our accelerated model has a
comparably fast speed as the "AlexNet", but is 4.7% more accurate.
