  Existing object proposal algorithms usually search for possible object
regions over multiple locations and scales separately, which ignore the
interdependency among different objects and deviate from the human perception
procedure. To incorporate global interdependency between objects into object
localization, we propose an effective Tree-structured Reinforcement Learning
(Tree-RL) approach to sequentially search for objects by fully exploiting both
the current observation and historical search paths. The Tree-RL approach
learns multiple searching policies through maximizing the long-term reward that
reflects localization accuracies over all the objects. Starting with taking the
entire image as a proposal, the Tree-RL approach allows the agent to
sequentially discover multiple objects via a tree-structured traversing scheme.
Allowing multiple near-optimal policies, Tree-RL offers more diversity in
search paths and is able to find multiple objects with a single feed-forward
pass. Therefore, Tree-RL can better cover different objects with various scales
which is quite appealing in the context of object proposal. Experiments on
PASCAL VOC 2007 and 2012 validate the effectiveness of the Tree-RL, which can
achieve comparable recalls with current object proposal algorithms via much
fewer candidate windows.
