  Numerous studies have revealed that deep learning-based medical image
classification models may exhibit bias towards specific demographic attributes,
such as race, gender, and age. Existing bias mitigation methods often achieve
high level of fairness at the cost of significant accuracy degradation. In
response to this challenge, we propose an innovative and adaptable Soft Nearest
Neighbor Loss-based channel pruning framework, which achieves fairness through
channel pruning. Traditionally, channel pruning is utilized to accelerate
neural network inference. However, our work demonstrates that pruning can also
be a potent tool for achieving fairness. Our key insight is that different
channels in a layer contribute differently to the accuracy of different groups.
By selectively pruning critical channels that lead to the accuracy difference
between the privileged and unprivileged groups, we can effectively improve
fairness without sacrificing accuracy significantly. Experiments conducted on
two skin lesion diagnosis datasets across multiple sensitive attributes
validate the effectiveness of our method in achieving state-of-the-art
trade-off between accuracy and fairness. Our code is available at
https://github.com/Kqp1227/Sensitive-Channel-Pruning.
