  One primary topic of multimodal learning is to jointly incorporate
heterogeneous information from different modalities. However most models often
suffer from unsatisfactory multimodal cooperation which cannot jointly utilize
all modalities well. Some methods are proposed to identify and enhance the
worse learnt modality but they are often hard to provide the fine-grained
observation of multimodal cooperation at sample-level with theoretical support.
Hence it is essential to reasonably observe and improve the fine-grained
cooperation between modalities especially when facing realistic scenarios where
the modality discrepancy could vary across different samples. To this end we
introduce a sample-level modality valuation metric to evaluate the contribution
of each modality for each sample. Via modality valuation we observe that
modality discrepancy indeed could be different at sample-level beyond the
global contribution discrepancy at dataset-level. We further analyze this issue
and improve cooperation between modalities at sample-level by enhancing the
discriminative ability of low-contributing modalities in a targeted manner.
Overall our methods reasonably observe the fine-grained uni-modal contribution
and achieve considerable improvement. The source code and dataset are available
at https://github.com/GeWu-Lab/Valuate-and-Enhance-Multimodal-Cooperation.
