  Real-world robotics systems deal with data from a multitude of modalities,
especially for tasks such as navigation and recognition. The performance of
those systems can drastically degrade when one or more modalities become
inaccessible, due to factors such as sensors' malfunctions or adverse
environments. Here, we argue modality hallucination as one effective way to
ensure consistent modality availability and thereby reduce unfavorable
consequences. While hallucinating data from a modality with richer information,
e.g., RGB to depth, has been researched extensively, we investigate the more
challenging low-to-high modality hallucination with interesting use cases in
robotics and autonomous systems. We present a novel hallucination architecture
that aggregates information from multiple fields of view of the local
neighborhood to recover the lost information from the extant modality. The
process is implemented by capturing a non-linear mapping between the data
modalities and the learned mapping is used to aid the extant modality to
mitigate the risk posed to the system in the adverse scenarios which involve
modality loss. We also conduct extensive classification and segmentation
experiments on UWRGBD and NYUD datasets and demonstrate that hallucination
allays the negative effects of the modality loss. Implementation and models:
https://github.com/kausic94/Hallucination
