  Federated learning (FL) has been recognized as a privacy-preserving
distributed machine learning paradigm that enables knowledge sharing among
various heterogeneous artificial intelligence (AIoT) devices through
centralized global model aggregation. FL suffers from model inaccuracy and slow
convergence due to the model heterogeneity of the AIoT devices involved.
Although various existing methods try to solve the bottleneck of the model
heterogeneity problem, most of them improve the accuracy of heterogeneous
models in a coarse-grained manner, which makes it still a great challenge to
deploy large-scale AIoT devices. To alleviate the negative impact of this
problem and take full advantage of the diversity of each heterogeneous model,
we propose an efficient framework named HierarchyFL, which uses a small amount
of public data for efficient and scalable knowledge across a variety of
differently structured models. By using self-distillation and our proposed
ensemble library, each hierarchical model can intelligently learn from each
other on cloud servers. Experimental results on various well-known datasets
show that HierarchyFL can not only maximize the knowledge sharing among various
heterogeneous models in large-scale AIoT systems, but also greatly improve the
model performance of each involved heterogeneous AIoT device.
