  This paper proposes an efficient federated distillation learning system
(EFDLS) for multi-task time series classification (TSC). EFDLS consists of a
central server and multiple mobile users, where different users may run
different TSC tasks. EFDLS has two novel components, namely a feature-based
student-teacher (FBST) framework and a distance-based weights matching (DBWM)
scheme. Within each user, the FBST framework transfers knowledge from its
teacher's hidden layers to its student's hidden layers via knowledge
distillation, with the teacher and student having identical network structure.
For each connected user, its student model's hidden layers' weights are
uploaded to the EFDLS server periodically. The DBWM scheme is deployed on the
server, with the least square distance used to measure the similarity between
the weights of two given models. This scheme finds a partner for each connected
user such that the user's and its partner's weights are the closest among all
the weights uploaded. The server exchanges and sends back the user's and its
partner's weights to these two users which then load the received weights to
their teachers' hidden layers. Experimental results show that the proposed
EFDLS achieves excellent performance on a set of selected UCR2018 datasets
regarding top-1 accuracy.
