  Deep reinforcement learning yields great results for a large array of
problems, but models are generally retrained anew for each new problem to be
solved. Prior learning and knowledge are difficult to incorporate when training
new models, requiring increasingly longer training as problems become more
complex. This is especially problematic for problems with sparse rewards. We
provide a solution to these problems by introducing Concept Network
Reinforcement Learning (CNRL), a framework which allows us to decompose
problems using a multi-level hierarchy. Concepts in a concept network are
reusable, and flexible enough to encapsulate feature extractors, skills, or
other concept networks. With this hierarchical learning approach, deep
reinforcement learning can be used to solve complex tasks in a modular way,
through problem decomposition. We demonstrate the strength of CNRL by training
a model to grasp a rectangular prism and precisely stack it on top of a cube
using a gripper on a Kinova JACO arm, simulated in MuJoCo. Our experiments show
that our use of hierarchy results in a 45x reduction in environment
interactions compared to the state-of-the-art on this task.
