  Despite recent progress in enhancing the privacy of federated learning (FL)
via differential privacy (DP), the trade-off of DP between privacy protection
and performance is still underexplored for real-world medical scenario. In this
paper, we propose to optimize the trade-off under the context of client-level
DP, which focuses on privacy during communications. However, FL for medical
imaging involves typically much fewer participants (hospitals) than other
domains (e.g., mobile devices), thus ensuring clients be differentially private
is much more challenging. To tackle this problem, we propose an adaptive
intermediary strategy to improve performance without harming privacy.
Specifically, we theoretically find splitting clients into sub-clients, which
serve as intermediaries between hospitals and the server, can mitigate the
noises introduced by DP without harming privacy. Our proposed approach is
empirically evaluated on both classification and segmentation tasks using two
public datasets, and its effectiveness is demonstrated with significant
performance improvements and comprehensive analytical studies. Code is
available at: https://github.com/med-air/Client-DP-FL.
