  In order for robots and other artificial agents to efficiently learn to
perform useful tasks defined by an end user, they must understand not only the
goals of those tasks, but also the structure and dynamics of that user's
environment. While existing work has looked at how the goals of a task can be
inferred from a human teacher, the agent is often left to learn about the
environment on its own. To address this limitation, we develop an algorithm,
Behavior Aware Modeling (BAM), which incorporates a teacher's knowledge into a
model of the transition dynamics of an agent's environment. We evaluate BAM
both in simulation and with real human teachers, learning from a combination of
task demonstrations and evaluative feedback, and show that it can outperform
approaches which do not explicitly consider this source of dynamics knowledge.
