  We humans can impeccably search for a target object, given its name only,
even in an unseen environment. We argue that this ability is largely due to
three main reasons: the incorporation of prior knowledge (or experience), the
adaptation of it to the new environment using the observed visual cues and most
importantly optimistically searching without giving up early. This is currently
missing in the state-of-the-art visual navigation methods based on
Reinforcement Learning (RL). In this paper, we propose to use externally
learned prior knowledge of the relative object locations and integrate it into
our model by constructing a neural graph. In order to efficiently incorporate
the graph without increasing the state-space complexity, we propose our
Graph-based Value Estimation (GVE) module. GVE provides a more accurate
baseline for estimating the Advantage function in actor-critic RL algorithm.
This results in reduced value estimation error and, consequently, convergence
to a more optimal policy. Through empirical studies, we show that our agent,
dubbed as the optimistic agent, has a more realistic estimate of the state
value during a navigation episode which leads to a higher success rate. Our
extensive ablation studies show the efficacy of our simple method which
achieves the state-of-the-art results measured by the conventional visual
navigation metrics, e.g. Success Rate (SR) and Success weighted by Path Length
(SPL), in AI2THOR environment.
