  Heterogeneous trajectory forecasting is critical for intelligent
transportation systems, but it is challenging because of the difficulty of
modeling the complex interaction relations among the heterogeneous road agents
as well as their agent-environment constraints. In this work, we propose a risk
and scene graph learning method for trajectory forecasting of heterogeneous
road agents, which consists of a Heterogeneous Risk Graph (HRG) and a
Hierarchical Scene Graph (HSG) from the aspects of agent category and their
movable semantic regions. HRG groups each kind of road agent and calculates
their interaction adjacency matrix based on an effective collision risk metric.
HSG of the driving scene is modeled by inferring the relationship between road
agents and road semantic layout aligned by the road scene grammar. Based on
this formulation, we can obtain effective trajectory forecasting in driving
situations, and superior performance to other state-of-the-art approaches is
demonstrated by exhaustive experiments on the nuScenes, ApolloScape, and
Argoverse datasets.
