  Acceleration of deep neural networks to meet a specific latency constraint is
essential for their deployment on mobile devices. In this paper, we design an
architecture aware latency constrained sparse (ALCS) framework to prune and
accelerate CNN models. Taking modern mobile computation architectures into
consideration, we propose Single Instruction Multiple Data (SIMD)-structured
pruning, along with a novel sparse convolution algorithm for efficient
computation. Besides, we propose to estimate the run time of sparse models with
piece-wise linear interpolation. The whole latency constrained pruning task is
formulated as a constrained optimization problem that can be efficiently solved
with Alternating Direction Method of Multipliers (ADMM). Extensive experiments
show that our system-algorithm co-design framework can achieve much better
Pareto frontier among network accuracy and latency on resource-constrained
mobile devices.
