  Many tasks in human environments require performing a sequence of navigation
and manipulation steps involving objects. In unstructured human environments,
the location and configuration of the objects involved often change in
unpredictable ways. This requires a high-level planning strategy that is robust
and flexible in an uncertain environment. We propose a novel dynamic planning
strategy, which can be trained from a set of example sequences. High level
tasks are expressed as a sequence of primitive actions or controllers (with
appropriate parameters). Our score function, based on Markov Random Field
(MRF), captures the relations between environment, controllers, and their
arguments. By expressing the environment using sets of attributes, the approach
generalizes well to unseen scenarios. We train the parameters of our MRF using
a maximum margin learning method. We provide a detailed empirical validation of
our overall framework demonstrating successful plan strategies for a variety of
tasks.
