  Motion prediction is highly relevant to the perception of dynamic objects and
static map elements in the scenarios of autonomous driving. In this work, we
propose PIP, the first end-to-end Transformer-based framework which jointly and
interactively performs online mapping, object detection and motion prediction.
PIP leverages map queries, agent queries and mode queries to encode the
instance-wise information of map elements, agents and motion intentions,
respectively. Based on the unified query representation, a differentiable
multi-task interaction scheme is proposed to exploit the correlation between
perception and prediction. Even without human-annotated HD map or agent's
historical tracking trajectory as guidance information, PIP realizes end-to-end
multi-agent motion prediction and achieves better performance than
tracking-based and HD-map-based methods. PIP provides comprehensive high-level
information of the driving scene (vectorized static map and dynamic objects
with motion information), and contributes to the downstream planning and
control. Code and models will be released for facilitating further research.
