  Teaching a deep reinforcement learning (RL) agent to follow instructions in
multi-task environments is a challenging problem. We consider that user defines
every task by a linear temporal logic (LTL) formula. However, some causal
dependencies in complex environments may be unknown to the user in advance.
Hence, when human user is specifying instructions, the robot cannot solve the
tasks by simply following the given instructions. In this work, we propose a
hierarchical reinforcement learning (HRL) framework in which a symbolic
transition model is learned to efficiently produce high-level plans that can
guide the agent efficiently solve different tasks. Specifically, the symbolic
transition model is learned by inductive logic programming (ILP) to capture
logic rules of state transitions. By planning over the product of the symbolic
transition model and the automaton derived from the LTL formula, the agent can
resolve causal dependencies and break a causally complex problem down into a
sequence of simpler low-level sub-tasks. We evaluate the proposed framework on
three environments in both discrete and continuous domains, showing advantages
over previous representative methods.
