  The promise of multimodal models for real-world applications has inspired
research in visualizing and understanding their internal mechanics with the end
goal of empowering stakeholders to visualize model behavior, perform model
debugging, and promote trust in machine learning models. However, modern
multimodal models are typically black-box neural networks, which makes it
challenging to understand their internal mechanics. How can we visualize the
internal modeling of multimodal interactions in these models? Our paper aims to
fill this gap by proposing MultiViz, a method for analyzing the behavior of
multimodal models by scaffolding the problem of interpretability into 4 stages:
(1) unimodal importance: how each modality contributes towards downstream
modeling and prediction, (2) cross-modal interactions: how different modalities
relate with each other, (3) multimodal representations: how unimodal and
cross-modal interactions are represented in decision-level features, and (4)
multimodal prediction: how decision-level features are composed to make a
prediction. MultiViz is designed to operate on diverse modalities, models,
tasks, and research areas. Through experiments on 8 trained models across 6
real-world tasks, we show that the complementary stages in MultiViz together
enable users to (1) simulate model predictions, (2) assign interpretable
concepts to features, (3) perform error analysis on model misclassifications,
and (4) use insights from error analysis to debug models. MultiViz is publicly
available, will be regularly updated with new interpretation tools and metrics,
and welcomes inputs from the community.
