  Multimodal learning seeks to utilize data from multiple sources to improve
the overall performance of downstream tasks. It is desirable for redundancies
in the data to make multimodal systems robust to missing or corrupted
observations in some correlated modalities. However, we observe that the
performance of several existing multimodal networks significantly deteriorates
if one or multiple modalities are absent at test time. To enable robustness to
missing modalities, we propose a simple and parameter-efficient adaptation
procedure for pretrained multimodal networks. In particular, we exploit
modulation of intermediate features to compensate for the missing modalities.
We demonstrate that such adaptation can partially bridge performance drop due
to missing modalities and outperform independent, dedicated networks trained
for the available modality combinations in some cases. The proposed adaptation
requires extremely small number of parameters (e.g., fewer than 1% of the total
parameters) and applicable to a wide range of modality combinations and tasks.
We conduct a series of experiments to highlight the missing modality robustness
of our proposed method on five different multimodal tasks across seven
datasets. Our proposed method demonstrates versatility across various tasks and
datasets, and outperforms existing methods for robust multimodal learning with
missing modalities.
