  Multimodal Deep Learning enhances decision-making by integrating diverse
information sources, such as texts, images, audio, and videos. To develop
trustworthy multimodal approaches, it is essential to understand how
uncertainty impacts these models. We introduce LUMA, a unique benchmark
dataset, featuring audio, image, and textual data from 50 classes, for learning
from uncertain and multimodal data. It extends the well-known CIFAR 10/100
dataset with audio samples extracted from three audio corpora, and text data
generated using the Gemma-7B Large Language Model (LLM). The LUMA dataset
enables the controlled injection of varying types and degrees of uncertainty to
achieve and tailor specific experiments and benchmarking initiatives. LUMA is
also available as a Python package including the functions for generating
multiple variants of the dataset with controlling the diversity of the data,
the amount of noise for each modality, and adding out-of-distribution samples.
A baseline pre-trained model is also provided alongside three uncertainty
quantification methods: Monte-Carlo Dropout, Deep Ensemble, and Reliable
Conflictive Multi-View Learning. This comprehensive dataset and its tools are
intended to promote and support the development and benchmarking of trustworthy
and robust multimodal deep learning approaches.
