  Despite progress in visual perception tasks such as image classification and
detection, computers still struggle to understand the interdependency of
objects in the scene as a whole, e.g., relations between objects or their
attributes. Existing methods often ignore global context cues capturing the
interactions among different object instances, and can only recognize a handful
of types by exhaustively training individual detectors for all possible
relationships. To capture such global interdependency, we propose a deep
Variation-structured Reinforcement Learning (VRL) framework to sequentially
discover object relationships and attributes in the whole image. First, a
directed semantic action graph is built using language priors to provide a rich
and compact representation of semantic correlations between object categories,
predicates, and attributes. Next, we use a variation-structured traversal over
the action graph to construct a small, adaptive action set for each step based
on the current state and historical actions. In particular, an ambiguity-aware
object mining scheme is used to resolve semantic ambiguity among object
categories that the object detector fails to distinguish. We then make
sequential predictions using a deep RL framework, incorporating global context
cues and semantic embeddings of previously extracted phrases in the state
vector. Our experiments on the Visual Relationship Detection (VRD) dataset and
the large-scale Visual Genome dataset validate the superiority of VRL, which
can achieve significantly better detection results on datasets involving
thousands of relationship and attribute types. We also demonstrate that VRL is
able to predict unseen types embedded in our action graph by learning
correlations on shared graph nodes.
