  One of the most important aspects of autonomous systems is safety. This
includes ensuring safe human-robot and safe robot-environment interaction when
autonomously performing complex tasks or in collaborative scenarios. Although
several methods have been introduced to tackle this, most are unsuitable for
real-time applications and require carefully hand-crafted obstacle
descriptions. In this work, we propose a method combining high-frequency and
real-time self and environment collision avoidance of a robotic manipulator
with low-frequency, multimodal, and high-resolution environmental perceptions
accumulated in a digital twin system. Our method is based on geometric
primitives, so-called primitive skeletons. These, in turn, are
information-compressed and real-time compatible digital representations of the
robot's body and environment, automatically generated from ultra-realistic
virtual replicas of the real world provided by the digital twin. Our approach
is a key enabler for closing the loop between environment perception and robot
control by providing the millisecond real-time control stage with a current and
accurate world description, empowering it to react to environmental changes. We
evaluate our whole-body collision avoidance on a 9-DOFs robot system through
five experiments, demonstrating the functionality and efficiency of our
framework.
