  Trajectory prediction and behavioral decision-making are two important tasks
for autonomous vehicles that require good understanding of the environmental
context; behavioral decisions are better made by referring to the outputs of
trajectory predictions. However, most current solutions perform these two tasks
separately. Therefore, a joint neural network that combines multiple cues is
proposed and named as the holistic transformer to predict trajectories and make
behavioral decisions simultaneously. To better explore the intrinsic
relationships between cues, the network uses existing knowledge and adopts
three kinds of attention mechanisms: the sparse multi-head type for reducing
noise impact, feature selection sparse type for optimally using partial prior
knowledge, and multi-head with sigmoid activation type for optimally using
posteriori knowledge. Compared with other trajectory prediction models, the
proposed model has better comprehensive performance and good interpretability.
Perceptual noise robustness experiments demonstrate that the proposed model has
good noise robustness. Thus, simultaneous trajectory prediction and behavioral
decision-making combining multiple cues can reduce computational costs and
enhance semantic relationships between scenes and agents.
