  Autonomous vehicles (AVs) need to share the road with multiple, heterogeneous
road users in a variety of driving scenarios. It is overwhelming and
unnecessary to carefully interact with all observed agents, and AVs need to
determine whether and when to interact with each surrounding agent. In order to
facilitate the design and testing of prediction and planning modules of AVs,
in-depth understanding of interactive behavior is expected with proper
representation, and events in behavior data need to be extracted and
categorized automatically. Answers to what are the essential patterns of
interactions are also crucial for these motivations in addition to answering
whether and when. Thus, learning to extract interactive driving events and
patterns from human data for tackling the whether-when-what tasks is of
critical importance for AVs. There is, however, no clear definition and
taxonomy of interactive behavior, and most of the existing works are based on
either manual labelling or hand-crafted rules and features. In this paper, we
propose the Interactive Driving event and pattern Extraction Network (IDE-Net),
which is a deep learning framework to automatically extract interaction events
and patterns directly from vehicle trajectories. In IDE-Net, we leverage the
power of multi-task learning and proposed three auxiliary tasks to assist the
pattern extraction in an unsupervised fashion. We also design a unique
spatial-temporal block to encode the trajectory data. Experimental results on
the INTERACTION dataset verified the effectiveness of such designs in terms of
better generalizability and effective pattern extraction. We find three
interpretable patterns of interactions, bringing insights for driver behavior
representation, modeling and comprehension. Both objective and subjective
evaluation metrics are adopted in our analysis of the learned patterns.
