  The task of dynamic scene graph generation (DynSGG) aims to generate scene
graphs for given videos, which involves modeling the spatial-temporal
information in the video. However, due to the long-tailed distribution of
samples in the dataset, previous DynSGG models fail to predict the tail
predicates. We argue that this phenomenon is due to previous methods that only
pay attention to the local spatial-temporal information and neglect the
consistency of multiple frames. To solve this problem, we propose a novel
DynSGG model based on multi-task learning, DynSGG-MTL, which introduces the
local interaction information and global human-action interaction information.
The interaction between objects and frame features makes the model more fully
understand the visual context of the single image. Long-temporal human actions
supervise the model to generate multiple scene graphs that conform to the
global constraints and avoid the model being unable to learn the tail
predicates. Extensive experiments on Action Genome dataset demonstrate the
efficacy of our proposed framework, which not only improves the dynamic scene
graph generation but also alleviates the long-tail problem.
