  Federated learning (FL) achieves collaborative learning without the need for
data sharing, thus preventing privacy leakage. To extend FL into a fully
decentralized algorithm, researchers have applied distributed optimization
algorithms to FL by considering machine learning (ML) tasks as parameter
optimization problems. Conversely, the consensus-based multi-hop federated
distillation (CMFD) proposed in the authors' previous work makes neural network
(NN) models get close with others in a function space rather than in a
parameter space. Hence, this study solves two unresolved challenges of CMFD:
(1) communication cost reduction and (2) visualization of model convergence.
Based on a proposed dynamic communication cost reduction method (DCCR), the
amount of data transferred in a network is reduced; however, with a slight
degradation in the prediction accuracy. In addition, a technique for
visualizing the distance between the NN models in a function space is also
proposed. The technique applies a dimensionality reduction technique by
approximating infinite-dimensional functions as numerical vectors to visualize
the trajectory of how the models change by the distributed learning algorithm.
