  Imitation Learning (IL) is an appealing approach to learn desirable
autonomous behavior. However, directing IL to achieve arbitrary goals is
difficult. In contrast, planning-based algorithms use dynamics models and
reward functions to achieve goals. Yet, reward functions that evoke desirable
behavior are often difficult to specify. In this paper, we propose Imitative
Models to combine the benefits of IL and goal-directed planning. Imitative
Models are probabilistic predictive models of desirable behavior able to plan
interpretable expert-like trajectories to achieve specified goals. We derive
families of flexible goal objectives, including constrained goal regions,
unconstrained goal sets, and energy-based goals. We show that our method can
use these objectives to successfully direct behavior. Our method substantially
outperforms six IL approaches and a planning-based approach in a dynamic
simulated autonomous driving task, and is efficiently learned from expert
demonstrations without online data collection. We also show our approach is
robust to poorly specified goals, such as goals on the wrong side of the road.
