  The tasks that an agent will need to solve often are not known during
training. However, if the agent knows which properties of the environment are
important then, after learning how its actions affect those properties, it may
be able to use this knowledge to solve complex tasks without training
specifically for them. Towards this end, we consider a setup in which an
environment is augmented with a set of user defined attributes that
parameterize the features of interest. We propose a method that learns a policy
for transitioning between "nearby" sets of attributes, and maintains a graph of
possible transitions. Given a task at test time that can be expressed in terms
of a target set of attributes, and a current state, our model infers the
attributes of the current state and searches over paths through attribute space
to get a high level plan, and then uses its low level policy to execute the
plan. We show in 3D block stacking, grid-world games, and StarCraft that our
model is able to generalize to longer, more complex tasks at test time by
composing simpler learned policies.
