  Federated learning (FL) allows a set of clients to collaboratively train a
machine-learning model without exposing local training samples. In this
context, it is considered to be privacy-preserving and hence has been adopted
by medical centers to train machine-learning models over private data. However,
in this paper, we propose a novel attack named MediLeak that enables a
malicious parameter server to recover high-fidelity patient images from the
model updates uploaded by the clients. MediLeak requires the server to generate
an adversarial model by adding a crafted module in front of the original model
architecture. It is published to the clients in the regular FL training process
and each client conducts local training on it to generate corresponding model
updates. Then, based on the FL protocol, the model updates are sent back to the
server and our proposed analytical method recovers private data from the
parameter updates of the crafted module. We provide a comprehensive analysis
for MediLeak and show that it can successfully break the state-of-the-art
cryptographic secure aggregation protocols, designed to protect the FL systems
from privacy inference attacks. We implement MediLeak on the MedMNIST and
COVIDx CXR-4 datasets. The results show that MediLeak can nearly perfectly
recover private images with high recovery rates and quantitative scores. We
further perform downstream tasks such as disease classification with the
recovered data, where our results show no significant performance degradation
compared to using the original training samples.
