  Perception of auditory events is inherently multimodal relying on both audio
and visual cues. A large number of existing multimodal approaches process each
modality using modality-specific models and then fuse the embeddings to encode
the joint information. In contrast, we employ heterogeneous graphs to
explicitly capture the spatial and temporal relationships between the
modalities and represent detailed information about the underlying signal.
Using heterogeneous graph approaches to address the task of visually-aware
acoustic event classification, which serves as a compact, efficient and
scalable way to represent data in the form of graphs. Through heterogeneous
graphs, we show efficiently modelling of intra- and inter-modality
relationships both at spatial and temporal scales. Our model can easily be
adapted to different scales of events through relevant hyperparameters.
Experiments on AudioSet, a large benchmark, shows that our model achieves
state-of-the-art performance.
