  Hierarchical Reinforcement Learning (HRL) has made notable progress in
complex control tasks by leveraging temporal abstraction. However, previous HRL
algorithms often suffer from serious data inefficiency as environments get
large. The extended components, $i.e.$, goal space and length of episodes,
impose a burden on either one or both high-level and low-level policies since
both levels share the total horizon of the episode. In this paper, we present a
method of Decoupling Horizons Using a Graph in Hierarchical Reinforcement
Learning (DHRL) which can alleviate this problem by decoupling the horizons of
high-level and low-level policies and bridging the gap between the length of
both horizons using a graph. DHRL provides a freely stretchable high-level
action interval, which facilitates longer temporal abstraction and faster
training in complex tasks. Our method outperforms state-of-the-art HRL
algorithms in typical HRL environments. Moreover, DHRL achieves long and
complex locomotion and manipulation tasks.
