  Large scale deep learning provides a tremendous opportunity to improve the
quality of content recommendation systems by employing both wider and deeper
models, but this comes at great infrastructural cost and carbon footprint in
modern data centers. Pruning is an effective technique that reduces both memory
and compute demand for model inference. However, pruning for online
recommendation systems is challenging due to the continuous data distribution
shift (a.k.a non-stationary data). Although incremental training on the full
model is able to adapt to the non-stationary data, directly applying it on the
pruned model leads to accuracy loss. This is because the sparsity pattern after
pruning requires adjustment to learn new patterns. To the best of our
knowledge, this is the first work to provide in-depth analysis and discussion
of applying pruning to online recommendation systems with non-stationary data
distribution. Overall, this work makes the following contributions: 1) We
present an adaptive dense to sparse paradigm equipped with a novel pruning
algorithm for pruning a large scale recommendation system with non-stationary
data distribution; 2) We design the pruning algorithm to automatically learn
the sparsity across layers to avoid repeating hand-tuning, which is critical
for pruning the heterogeneous architectures of recommendation systems trained
with non-stationary data.
