  The rise in loosely-structured data available through text, images, and other
modalities has called for new ways of querying them. Multimedia Information
Retrieval has filled this gap and has witnessed exciting progress in recent
years. Tasks such as search and retrieval of extensive multimedia archives have
undergone massive performance improvements, driven to a large extent by recent
developments in multimodal deep learning. However, methods in this field remain
limited in the kinds of queries they support and, in particular, their
inability to answer database-like queries. For this reason, inspired by recent
work on neural databases, we propose a new framework, which we name Multimodal
Neural Databases (MMNDBs). MMNDBs can answer complex database-like queries that
involve reasoning over different input modalities, such as text and images, at
scale. In this paper, we present the first architecture able to fulfill this
set of requirements and test it with several baselines, showing the limitations
of currently available models. The results show the potential of these new
techniques to process unstructured data coming from different modalities,
paving the way for future research in the area. Code to replicate the
experiments will be released at
https://github.com/GiovanniTRA/MultimodalNeuralDatabases
