  Learning to solve long horizon temporally extended tasks with reinforcement
learning has been a challenge for several years now. We believe that it is
important to leverage both the hierarchical structure of complex tasks and to
use expert supervision whenever possible to solve such tasks. This work
introduces an interpretable hierarchical agent framework by combining planning
and semantic goal directed reinforcement learning. We assume access to certain
spatial and haptic predicates and construct a simple and powerful semantic goal
space. These semantic goal representations are more interpretable, making
expert supervision and intervention easier. They also eliminate the need to
write complex, dense reward functions thereby reducing human engineering
effort. We evaluate our framework on a robotic block manipulation task and show
that it performs better than other methods, including both sparse and dense
reward functions. We also suggest some next steps and discuss how this
framework makes interaction and collaboration with humans easier.
