  This paper uses robots to assemble pegs into holes on surfaces with different
colors and textures. It especially targets at the problem of peg-in-hole
assembly with initial position uncertainty. Two in-hand cameras and a
force-torque sensor are used to account for the position uncertainty. A program
sequence comprising learning-based visual servoing, spiral search, and
impedance control is implemented to perform the peg-in-hole task with feedback
from the above sensors. Contributions are mainly made in the learning-based
visual servoing of the sequence, where a deep neural network is trained with
various sets of synthetic data generated using the concept of domain
randomization to predict where a hole is. In the experiments and analysis
section, the network is analyzed and compared, and a real-world robotic system
to insert pegs to holes using the proposed method is implemented. The results
show that the implemented peg-in-hole assembly system can perform successful
peg-in-hole insertions on surfaces with various colors and textures. It can
generally speed up the entire peg-in-hole process.
