  In this paper, we propose a Similarity-Quality-based Messenger Distillation
(SQMD) framework for heterogeneous asynchronous on-device healthcare analytics.
By introducing a preloaded reference dataset, SQMD enables all participant
devices to distill knowledge from peers via messengers (i.e., the soft labels
of the reference dataset generated by clients) without assuming the same model
architecture. Furthermore, the messengers also carry important auxiliary
information to calculate the similarity between clients and evaluate the
quality of each client model, based on which the central server creates and
maintains a dynamic collaboration graph (communication graph) to improve the
personalization and reliability of SQMD under asynchronous conditions.
Extensive experiments on three real-life datasets show that SQMD achieves
superior performance.
