  Deep neural networks have demonstrated remarkable performance in various
tasks. With a growing need for sparse deep learning, model compression
techniques, especially pruning, have gained significant attention. However,
conventional pruning techniques can inadvertently exacerbate algorithmic bias,
resulting in unequal predictions. To address this, we define a fair pruning
task where a sparse model is derived subject to fairness requirements. In
particular, we propose a framework to jointly optimize the pruning mask and
weight update processes with fairness constraints. This framework is engineered
to compress models that maintain performance while ensuring fairness in a
single execution. To this end, we formulate the fair pruning problem as a novel
constrained bi-level optimization task and derive efficient and effective
solving strategies. We design experiments spanning various datasets and
settings to validate our proposed method. Our empirical analysis contrasts our
framework with several mainstream pruning strategies, emphasizing our method's
superiority in maintaining model fairness, performance, and efficiency.
