  Model compression by way of parameter pruning, quantization, or distillation
has recently gained popularity as an approach for reducing the computational
requirements of modern deep neural network models for NLP. Inspired by prior
works suggesting a connection between simpler, more generalizable models and
those that lie within wider loss basins, we hypothesize that optimizing for
flat minima should lead to simpler parameterizations and thus more compressible
models. We propose to combine sharpness-aware minimization (SAM) with various
task-specific model compression methods, including iterative magnitude pruning
(IMP), structured pruning with a distillation objective, and post-training
dynamic quantization. Empirically, we show that optimizing for flatter minima
consistently leads to greater compressibility of parameters compared to vanilla
Adam when fine-tuning BERT models, with little to no loss in accuracy on the
GLUE text classification and SQuAD question answering benchmarks. Moreover, SAM
finds superior winning tickets during IMP that 1) are amenable to vanilla Adam
optimization, and 2) transfer more effectively across tasks.
