  We present a novel two-layer hierarchical reinforcement learning approach
equipped with a Goals Relational Graph (GRG) for tackling the partially
observable goal-driven task, such as goal-driven visual navigation. Our GRG
captures the underlying relations of all goals in the goal space through a
Dirichlet-categorical process that facilitates: 1) the high-level network
raising a sub-goal towards achieving a designated final goal; 2) the low-level
network towards an optimal policy; and 3) the overall system generalizing
unseen environments and goals. We evaluate our approach with two settings of
partially observable goal-driven tasks -- a grid-world domain and a robotic
object search task. Our experimental results show that our approach exhibits
superior generalization performance on both unseen environments and new goals.
