  Federated learning, which shares the weights of the neural network across
clients, is gaining attention in the healthcare sector as it enables training
on a large corpus of decentralized data while maintaining data privacy. For
example, this enables neural network training for COVID-19 diagnosis on chest
X-ray (CXR) images without collecting patient CXR data across multiple
hospitals. Unfortunately, the exchange of the weights quickly consumes the
network bandwidth if highly expressive network architecture is employed.
So-called split learning partially solves this problem by dividing a neural
network into a client and a server part, so that the client part of the network
takes up less extensive computation resources and bandwidth. However, it is not
clear how to find the optimal split without sacrificing the overall network
performance. To amalgamate these methods and thereby maximize their distinct
strengths, here we show that the Vision Transformer, a recently developed deep
learning architecture with straightforward decomposable configuration, is
ideally suitable for split learning without sacrificing performance. Even under
the non-independent and identically distributed data distribution which
emulates a real collaboration between hospitals using CXR datasets from
multiple sources, the proposed framework was able to attain performance
comparable to data-centralized training. In addition, the proposed framework
along with heterogeneous multi-task clients also improves individual task
performances including the diagnosis of COVID-19, eliminating the need for
sharing large weights with innumerable parameters. Our results affirm the
suitability of Transformer for collaborative learning in medical imaging and
pave the way forward for future real-world implementations.
