  We propose an imitation learning system for autonomous driving in urban
traffic with interactions. We train a Behavioral Cloning~(BC) policy to imitate
driving behavior collected from the real urban traffic, and apply the data
aggregation algorithm to improve its performance iteratively. Applying data
aggregation in this setting comes with two challenges. The first challenge is
that it is expensive and dangerous to collect online rollout data in the real
urban traffic. Creating similar traffic scenarios in simulator like CARLA for
online rollout collection can also be difficult. Instead, we propose to create
a weak simulator from the training dataset, in which all the surrounding
vehicles follow the data trajectory provided by the dataset. We find that the
collected online data in such a simulator can still be used to improve BC
policy's performance. The second challenge is the tedious and time-consuming
process of human labelling process during online rollout. To solve this
problem, we use an A$^*$ planner as a pseudo-expert to provide expert-like
demonstration. We validate our proposed imitation learning system in the real
urban traffic scenarios. The experimental results show that our system can
significantly improve the performance of baseline BC policy.
