  Language model debiasing has emerged as an important field of study in the
NLP community. Numerous debiasing techniques were proposed, but bias ablation
remains an unaddressed issue. We demonstrate a novel framework for inspecting
bias in pre-trained transformer-based language models via movement pruning.
Given a model and a debiasing objective, our framework finds a subset of the
model containing less bias than the original model. We implement our framework
by pruning the model while fine-tuning it on the debiasing objective. Optimized
are only the pruning scores - parameters coupled with the model's weights that
act as gates. We experiment with pruning attention heads, an important building
block of transformers: we prune square blocks, as well as establish a new way
of pruning the entire heads. Lastly, we demonstrate the usage of our framework
using gender bias, and based on our findings, we propose an improvement to an
existing debiasing method. Additionally, we re-discover a bias-performance
trade-off: the better the model performs, the more bias it contains.
