  Federated learning (FL) faces three major difficulties: cross-domain,
heterogeneous models, and non-i.i.d. labels scenarios. Existing FL methods fail
to handle the above three constraints at the same time, and the level of
privacy protection needs to be lowered (e.g., the model architecture and data
category distribution can be shared). In this work, we propose the challenging
"completely heterogeneous" scenario in FL, which refers to that each client
will not expose any private information including feature space, model
architecture, and label distribution. We then devise an FL framework based on
parameter decoupling and data-free knowledge distillation to solve the problem.
Experiments show that our proposed method achieves high performance in
completely heterogeneous scenarios where other approaches fail.
