  Developing robust artificial intelligence (AI) models that generalize well to
unseen datasets is challenging and usually requires large and variable
datasets, preferably from multiple institutions. In federated learning (FL), a
model is trained collaboratively at numerous sites that hold local datasets
without exchanging them. So far, the impact of training strategy, i.e., local
versus collaborative, on the diagnostic on-domain and off-domain performance of
AI models interpreting chest radiographs has not been assessed. Consequently,
using 610,000 chest radiographs from five institutions across the globe, we
assessed diagnostic performance as a function of training strategy (i.e., local
vs. collaborative), network architecture (i.e., convolutional vs.
transformer-based), generalization performance (i.e., on-domain vs.
off-domain), imaging finding (i.e., cardiomegaly, pleural effusion, pneumonia,
atelectasis, consolidation, pneumothorax, and no abnormality), dataset size
(i.e., from n=18,000 to 213,921 radiographs), and dataset diversity. Large
datasets not only showed minimal performance gains with FL but, in some
instances, even exhibited decreases. In contrast, smaller datasets revealed
marked improvements. Thus, on-domain performance was mainly driven by training
data size. However, off-domain performance leaned more on training diversity.
When trained collaboratively across diverse external institutions, AI models
consistently surpassed models trained locally for off-domain tasks, emphasizing
FL's potential in leveraging data diversity. In conclusion, FL can bolster
diagnostic privacy, reproducibility, and off-domain reliability of AI models
and, potentially, optimize healthcare outcomes.
