  In this work we propose a learning approach to high-precision robotic
assembly problems. We focus on the contact-rich phase, where the assembly
pieces are in close contact with each other. Unlike many learning-based
approaches that heavily rely on vision or spatial tracking, our approach takes
force/torque in task space as the only observation. Our training environment is
robotless, as the end-effector is not attached to any specific robot. Trained
policies can then be applied to different robotic arms without re-training.
This approach can greatly reduce complexity to perform contact-rich robotic
assembly in the real world, especially in unstructured settings such as in
architectural construction. To achieve it, we have developed a new distributed
RL agent, named Recurrent Distributed DDPG (RD2), which extends Ape-X DDPG with
recurrency and makes two structural improvements on prioritized experience
replay. Our results show that RD2 is able to solve two fundamental
high-precision assembly tasks, lap-joint and peg-in-hole, and outperforms two
state-of-the-art algorithms, Ape-X DDPG and PPO with LSTM. We have successfully
evaluated our robot-agnostic policies on three robotic arms, Kuka KR60, Franka
Panda, and UR10, in simulation. The video presenting our experiments is
available at https://sites.google.com/view/rd2-rl
