  Heterogeneous graphs provide a compact, efficient, and scalable way to model
data involving multiple disparate modalities. This makes modeling audiovisual
data using heterogeneous graphs an attractive option. However, graph structure
does not appear naturally in audiovisual data. Graphs for audiovisual data are
constructed manually which is both difficult and sub-optimal. In this work, we
address this problem by (i) proposing a parametric graph construction strategy
for the intra-modal edges, and (ii) learning the crossmodal edges. To this end,
we develop a new model, heterogeneous graph crossmodal network (HGCN) that
learns the crossmodal edges. Our proposed model can adapt to various spatial
and temporal scales owing to its parametric construction, while the learnable
crossmodal edges effectively connect the relevant nodes across modalities.
Experiments on a large benchmark dataset (AudioSet) show that our model is
state-of-the-art (0.53 mean average precision), outperforming transformer-based
models and other graph-based models.
