  Intelligent manipulation benefits from the capacity to flexibly control an
end-effector with high degrees of freedom (DoF) and dynamically react to the
environment. However, due to the challenges of collecting effective training
data and learning efficiently, most grasping algorithms today are limited to
top-down movements and open-loop execution. In this work, we propose a new
low-cost hardware interface for collecting grasping demonstrations by people in
diverse environments. Leveraging this data, we show that it is possible to
train a robust end-to-end 6DoF closed-loop grasping model with reinforcement
learning that transfers to real robots. A key aspect of our grasping model is
that it uses "action-view" based rendering to simulate future states with
respect to different possible actions. By evaluating these states using a
learned value function (Q-function), our method is able to better select
corresponding actions that maximize total rewards (i.e., grasping success). Our
final grasping system is able to achieve reliable 6DoF closed-loop grasping of
novel objects across various scene configurations, as well as dynamic scenes
with moving objects.
