  We consider task and motion planning in complex dynamic environments for
problems expressed in terms of a set of Linear Temporal Logic (LTL)
constraints, and a reward function. We propose a methodology based on
reinforcement learning that employs deep neural networks to learn low-level
control policies as well as task-level option policies. A major challenge in
this setting, both for neural network approaches and classical planning, is the
need to explore future worlds of a complex and interactive environment. To this
end, we integrate Monte Carlo Tree Search with hierarchical neural net control
policies trained on expressive LTL specifications. This paper investigates the
ability of neural networks to learn both LTL constraints and control policies
in order to generate task plans in complex environments. We demonstrate our
approach in a simulated autonomous driving setting, where a vehicle must drive
down a road in traffic, avoid collisions, and navigate an intersection, all
while obeying given rules of the road.
