  Recent work on the lottery ticket hypothesis has produced highly sparse
Transformers for NMT while maintaining BLEU. However, it is unclear how such
pruning techniques affect a model's learned representations. By probing
Transformers with more and more low-magnitude weights pruned away, we find that
complex semantic information is first to be degraded. Analysis of internal
activations reveals that higher layers diverge most over the course of pruning,
gradually becoming less complex than their dense counterparts. Meanwhile, early
layers of sparse models begin to perform more encoding. Attention mechanisms
remain remarkably consistent as sparsity increases.
