  This paper proposes a novel approach that enables a robot to learn an
objective function incrementally from human directional corrections. Existing
methods learn from human magnitude corrections; since a human needs to
carefully choose the magnitude of each correction, those methods can easily
lead to over-corrections and learning inefficiency. The proposed method only
requires human directional corrections -- corrections that only indicate the
direction of an input change without indicating its magnitude. We only assume
that each correction, regardless of its magnitude, points in a direction that
improves the robot's current motion relative to an unknown objective function.
The allowable corrections satisfying this assumption account for half of the
input space, as opposed to the magnitude corrections which have to lie in a
shrinking level set. For each directional correction, the proposed method
updates the estimate of the objective function based on a cutting plane method,
which has a geometric interpretation. We have established theoretical results
to show the convergence of the learning process. The proposed method has been
tested in numerical examples, a user study on two human-robot games, and a
real-world quadrotor experiment. The results confirm the convergence of the
proposed method and further show that the method is significantly more
effective (higher success rate), efficient/effortless (less human corrections
needed), and potentially more accessible (fewer early wasted trials) than the
state-of-the-art robot learning frameworks.
