  We present an algorithm for combining natural language processing (NLP) and
fast robot motion planning to automatically generate robot movements. Our
formulation uses a novel concept called Dynamic Constraint Mapping to transform
complex, attribute-based natural language instructions into appropriate cost
functions and parametric constraints for optimization-based motion planning. We
generate a factor graph from natural language instructions called the Dynamic
Grounding Graph (DGG), which takes latent parameters into account. The
coefficients of this factor graph are learned based on conditional random
fields (CRFs) and are used to dynamically generate the constraints for motion
planning. We map the cost function directly to the motion parameters of the
planner and compute smooth trajectories in dynamic scenes. We highlight the
performance of our approach in a simulated environment and via a human
interacting with a 7-DOF Fetch robot using intricate language commands
including negation, orientation specification, and distance constraints.
