  To ensure the safety and efficiency of its maneuvers, an Autonomous Vehicle
(AV) should anticipate the future intentions of surrounding vehicles using its
sensor information. If an AV can predict its surrounding vehicles' future
trajectories, it can make safe and efficient manoeuvre decisions. In this
paper, we present such a Deep Spatio-Temporal Context-Aware decision Network
(DST-CAN) model for predictive manoeuvre planning of AVs. A memory neuron
network is used to predict future trajectories of its surrounding vehicles. The
driving environment's spatio-temporal information (past, present, and predicted
future trajectories) are embedded into a context-aware grid. The proposed
DST-CAN model employs these context-aware grids as inputs to a convolutional
neural network to understand the spatial relationships between the vehicles and
determine a safe and efficient manoeuvre decision. The DST-CAN model also uses
information of human driving behavior on a highway. Performance evaluation of
DST-CAN has been carried out using two publicly available NGSIM US-101 and I-80
datasets. Also, rule-based ground truth decisions have been compared with those
generated by DST-CAN. The results clearly show that DST-CAN can make much
better decisions with 3-sec of predicted trajectories of neighboring vehicles
compared to currently existing methods that do not use this prediction.
