  Vision-based robotic assembly is a crucial yet challenging task as the
interaction with multiple objects requires high levels of precision. In this
paper, we propose an integrated 6D robotic system to perceive, grasp,
manipulate and assemble blocks with tight tolerances. Aiming to provide an
off-the-shelf RGB-only solution, our system is built upon a monocular 6D object
pose estimation network trained solely with synthetic images leveraging
physically-based rendering. Subsequently, pose-guided 6D transformation along
with collision-free assembly is proposed to construct any designed structure
with arbitrary initial poses. Our novel 3-axis calibration operation further
enhances the precision and robustness by disentangling 6D pose estimation and
robotic assembly. Both quantitative and qualitative results demonstrate the
effectiveness of our proposed 6D robotic assembly system.
