  Technological advances facilitate the ability to acquire multimodal data,
posing a challenge for recognition systems while also providing an opportunity
to use the heterogeneous nature of the information to increase the
generalization capability of models. An often overlooked issue is the cost of
the labeling process, which is typically high due to the need for a significant
investment in time and money associated with human experts. Existing
semi-supervised learning methods often focus on operating in the feature space
created by the fusion of available modalities, neglecting the potential for
cross-utilizing complementary information available in each modality. To
address this problem, we propose Cross-Modality Clustering-based Self-Labeling
(CMCSL). Based on a small set of pre-labeled data, CMCSL groups instances
belonging to each modality in the deep feature space and then propagates known
labels within the resulting clusters. Next, information about the instances'
class membership in each modality is exchanged based on the Euclidean distance
to ensure more accurate labeling. Experimental evaluation conducted on 20
datasets derived from the MM-IMDb dataset indicates that cross-propagation of
labels between modalities -- especially when the number of pre-labeled
instances is small -- can allow for more reliable labeling and thus increase
the classification performance in each modality.
