  Using sensor data from multiple modalities presents an opportunity to encode
redundant and complementary features that can be useful when one modality is
corrupted or noisy. Humans do this everyday, relying on touch and
proprioceptive feedback in visually-challenging environments. However, robots
might not always know when their sensors are corrupted, as even broken sensors
can return valid values. In this work, we introduce the Crossmodal Compensation
Model (CCM), which can detect corrupted sensor modalities and compensate for
them. CMM is a representation model learned with self-supervision that
leverages unimodal reconstruction loss for corruption detection. CCM then
discards the corrupted modality and compensates for it with information from
the remaining sensors. We show that CCM learns rich state representations that
can be used for contact-rich manipulation policies, even when input modalities
are corrupted in ways not seen during training time.
