  To determine the 3D orientation and 3D location of objects in the
surroundings of a camera mounted on a robot or mobile device, we developed two
powerful algorithms in object detection and temporal tracking that are combined
seamlessly for robotic perception and interaction as well as Augmented Reality
(AR). A separate evaluation of, respectively, the object detection and the
temporal tracker demonstrates the important stride in research as well as the
impact on industrial robotic applications and AR. When evaluated on a standard
dataset, the detector produced the highest f1-score with a large margin while
the tracker generated the best accuracy at a very low latency of approximately
2 ms per frame with one CPU core: both algorithms outperforming the state of
the art. When combined, we achieve a powerful framework that is robust to
handle multiple instances of the same object under occlusion and clutter while
attaining real-time performance. Aiming at stepping beyond the simple scenarios
used by current systems, often constrained by having a single object in absence
of clutter, averting to touch the object to prevent close-range partial
occlusion, selecting brightly colored objects to easily segment them
individually or assuming that the object has simple geometric structure, we
demonstrate the capacity to handle challenging cases under clutter, partial
occlusion and varying lighting conditions with objects of different shapes and
sizes.
