  This paper details our winning submission to Phase 1 of the 2021 Real Robot
Challenge; a challenge in which a three-fingered robot must carry a cube along
specified goal trajectories. To solve Phase 1, we use a pure reinforcement
learning approach which requires minimal expert knowledge of the robotic
system, or of robotic grasping in general. A sparse, goal-based reward is
employed in conjunction with Hindsight Experience Replay to teach the control
policy to move the cube to the desired x and y coordinates of the goal.
Simultaneously, a dense distance-based reward is employed to teach the policy
to lift the cube to the z coordinate (the height component) of the goal. The
policy is trained in simulation with domain randomisation before being
transferred to the real robot for evaluation. Although performance tends to
worsen after this transfer, our best policy can successfully lift the real cube
along goal trajectories via an effective pinching grasp. Our approach
outperforms all other submissions, including those leveraging more traditional
robotic control techniques, and is the first pure learning-based method to
solve this challenge.
