  Heterogeneous federated multi-task learning (HFMTL) is a federated learning
technique that combines heterogeneous tasks of different clients to achieve
more accurate, comprehensive predictions. In real-world applications, visual
and natural language tasks typically require large-scale models to extract
high-level abstract features. However, large-scale models cannot be directly
applied to existing federated multi-task learning methods. Existing HFML
methods also disregard the impact of gradient conflicts on multi-task
optimization during the federated aggregation process. In this work, we propose
an innovative framework called FedBone, which enables the construction of
large-scale models with better generalization from the perspective of
server-client split learning and gradient projection. We split the entire model
into two components: a large-scale general model (referred to as the general
model) on the cloud server and multiple task-specific models (referred to as
the client model) on edge clients, solving the problem of insufficient
computing power on edge clients. The conflicting gradient projection technique
is used to enhance the generalization of the large-scale general model between
different tasks. The proposed framework is evaluated on two benchmark datasets
and a real ophthalmic dataset. Comprehensive results demonstrate that FedBone
efficiently adapts to heterogeneous local tasks of each client and outperforms
existing federated learning algorithms in most dense prediction and
classification tasks with off-the-shelf computational resources on the client
side.
