  Sequence-to-sequence language models can be used to produce abstractive
summaries which are coherent, relevant, and concise. Still, model sizes can
make deployment in latency-sensitive or web-scale implementations difficult.
This paper studies the relationship between model size, structured pruning,
inference efficiency, and summarization accuracy on widely used summarization
datasets. We show that model accuracy is tied to the encoder size while
inference efficiency is connected to the decoder. Using asymmetric pruning can
lead to nearly 3x improvement in inference latency with ~1 point loss in
Rouge-2. Moreover, we find both the average degradation and the role of
asymmetry to be consistent across model sizes and variations in datasets.
