  Decentralized learning is widely employed for collaboratively training models
using distributed data over wireless networks. Existing decentralized learning
methods primarily focus on training single-modal networks. For the
decentralized multi-modal learning (DMML), the modality heterogeneity and the
non-independent and non-identically distributed (non-IID) data across devices
make it difficult for the training model to capture the correlated features
across different modalities. Moreover, modality competition can result in
training imbalance among different modalities, which can significantly impact
the performance of DMML. To improve the training performance in the presence of
non-IID data and modality heterogeneity, we propose a novel DMML with knowledge
distillation (DMML-KD) framework, which decomposes the extracted feature into
the modality-common and the modality-specific components. In the proposed
DMML-KD, a generator is applied to learn the global conditional distribution of
the modality-common features, thereby guiding the modality-common features of
different devices towards the same distribution. Meanwhile, we propose to
decrease the number of local iterations for the modalities with fast training
speed in DMML-KD to address the imbalanced training. We design a balance metric
based on the parameter variation to evaluate the training speed of different
modalities in DMML-KD. Using this metric, we optimize the number of local
iterations for different modalities on each device under the constraint of
remaining energy on devices. Experimental results demonstrate that the proposed
DMML-KD with training balance can effectively improve the training performance
of DMML.
