  Most successes in robotic manipulation have been restricted to single-arm
gripper robots, whose low dexterity limits the range of solvable tasks to
pick-and-place, inser-tion, and object rearrangement. More complex tasks such
as assembly require dual and multi-arm platforms, but entail a suite of unique
challenges such as bi-arm coordination and collision avoidance, robust
grasping, and long-horizon planning. In this work we investigate the
feasibility of training deep reinforcement learning (RL) policies in simulation
and transferring them to the real world (Sim2Real) as a generic methodology for
obtaining performant controllers for real-world bi-manual robotic manipulation
tasks. As a testbed for bi-manual manipulation, we develop the U-Shape Magnetic
BlockAssembly Task, wherein two robots with parallel grippers must connect 3
magnetic blocks to form a U-shape. Without manually-designed controller nor
human demonstrations, we demonstrate that with careful Sim2Real considerations,
our policies trained with RL in simulation enable two xArm6 robots to solve the
U-shape assembly task with a success rate of above90% in simulation, and 50% on
real hardware without any additional real-world fine-tuning. Through careful
ablations,we highlight how each component of the system is critical for such
simple and successful policy learning and transfer,including task
specification, learning algorithm, direct joint-space control, behavior
constraints, perception and actuation noises, action delays and action
interpolation. Our results present a significant step forward for bi-arm
capability on real hardware, and we hope our system can inspire future research
on deep RL and Sim2Real transfer of bi-manualpolicies, drastically scaling up
the capability of real-world robot manipulators.
