  In the evolving application of medical artificial intelligence, federated
learning is notable for its ability to protect training data privacy. Federated
learning facilitates collaborative model development without the need to share
local data from healthcare institutions. Yet, the statistical and system
heterogeneity among these institutions poses substantial challenges, which
affects the effectiveness of federated learning and hampers the exchange of
information between clients. To address these issues, we introduce a novel
approach, MH-pFLGB, which employs a global bypass strategy to mitigate the
reliance on public datasets and navigate the complexities of non-IID data
distributions. Our method enhances traditional federated learning by
integrating a global bypass model, which would share the information among the
clients, but also serves as part of the network to enhance the performance on
each client. Additionally, MH-pFLGB provides a feature fusion module to better
combine the local and global features. We validate \model{}'s effectiveness and
adaptability through extensive testing on different medical tasks,
demonstrating superior performance compared to existing state-of-the-art
methods.
