  Model sparsification in deep learning promotes simpler, more interpretable
models with fewer parameters. This not only reduces the model's memory
footprint and computational needs but also shortens inference time. This work
focuses on creating sparse models optimized for multiple tasks with fewer
parameters. These parsimonious models also possess the potential to match or
outperform dense models in terms of performance. In this work, we introduce
channel-wise l1/l2 group sparsity in the shared convolutional layers parameters
(or weights) of the multi-task learning model. This approach facilitates the
removal of extraneous groups i.e., channels (due to l1 regularization) and also
imposes a penalty on the weights, further enhancing the learning efficiency for
all tasks (due to l2 regularization). We analyzed the results of group sparsity
in both single-task and multi-task settings on two widely-used Multi-Task
Learning (MTL) datasets: NYU-v2 and CelebAMask-HQ. On both datasets, which
consist of three different computer vision tasks each, multi-task models with
approximately 70% sparsity outperform their dense equivalents. We also
investigate how changing the degree of sparsification influences the model's
performance, the overall sparsity percentage, the patterns of sparsity, and the
inference time.
