  It can be difficult to autonomously produce driver behavior so that it
appears natural to other traffic participants. Through Inverse Reinforcement
Learning (IRL), we can automate this process by learning the underlying reward
function from human demonstrations. We propose a new IRL algorithm that learns
a goal-conditioned spatiotemporal reward function. The resulting costmap is
used by Model Predictive Controllers (MPCs) to perform a task without any
hand-designing or hand-tuning of the cost function. We evaluate our proposed
Goal-conditioned SpatioTemporal Zeroing Maximum Entropy Deep IRL (GSTZ)-MEDIRL
framework together with MPC in the CARLA simulator for autonomous driving, lane
keeping, and lane changing tasks in a challenging dense traffic highway
scenario. Our proposed methods show higher success rates compared to other
baseline methods including behavior cloning, state-of-the-art RL policies, and
MPC with a learning-based behavior prediction model.
