  In this paper, we present a multi-object 6D detection and tracking pipeline
for potentially similar and non-textured objects. The combination of a
convolutional neural network for object classification and rough pose
estimation with a local pose refinement and an automatic mismatch detection
enables direct application in real-time AR scenarios. A new network
architecture, trained solely with synthetic images, allows simultaneous pose
estimation of multiple objects with reduced GPU memory consumption and enhanced
performance. In addition, the pose estimates are further improved by a local
edge-based refinement step that explicitly exploits known object geometry
information. For continuous movements, the sole use of local refinement reduces
pose mismatches due to geometric ambiguities or occlusions. We showcase the
entire tracking pipeline and demonstrate the benefits of the combined approach.
Experiments on a challenging set of non-textured similar objects demonstrate
the enhanced quality compared to the baseline method. Finally, we illustrate
how the system can be used in a real AR assistance application within the field
of construction.
