Hierarchical Reinforcement Learning with Deep Nested Agents
  Deep hierarchical reinforcement learning has gained a lot of attention in
recent years due to its ability to produce state-of-the-art results in
challenging environments where non-hierarchical frameworks fail to learn useful
policies. However, as problem domains become more complex, deep hierarchical
reinforcement learning can become inefficient, leading to longer convergence
times and poor performance. We introduce the Deep Nested Agent framework, which
is a variant of deep hierarchical reinforcement learning where information from
the main agent is propagated to the low level $nested$ agent by incorporating
this information into the nested agent's state. We demonstrate the
effectiveness and performance of the Deep Nested Agent framework by applying it
to three scenarios in Minecraft with comparisons to a deep non-hierarchical
single agent framework, as well as, a deep hierarchical framework.
