  Federated learning (FL) aims to collaboratively learn deep learning model
parameters from decentralized data archives (i.e., clients) without accessing
training data on clients. However, the training data across clients might be
not independent and identically distributed (non-IID), which may result in
difficulty in achieving optimal model convergence. In this work, we investigate
the capability of state-of-the-art transformer architectures (which are
MLP-Mixer, ConvMixer, PoolFormer) to address the challenges related to non-IID
training data across various clients in the context of FL for multi-label
classification (MLC) problems in remote sensing (RS). The considered
transformer architectures are compared among themselves and with the ResNet-50
architecture in terms of their: 1) robustness to training data heterogeneity;
2) local training complexity; and 3) aggregation complexity under different
non-IID levels. The experimental results obtained on the BigEarthNet-S2
benchmark archive demonstrate that the considered architectures increase the
generalization ability with the cost of higher local training and aggregation
complexities. On the basis of our analysis, some guidelines are derived for a
proper selection of transformer architecture in the context of FL for RS MLC.
The code of this work is publicly available at
https://git.tu-berlin.de/rsim/FL-Transformer.
