  In this paper, we investigate the problems that Convolutional Neural Networks
(CNN)-based pose estimators have with symmetric objects. We considered the
value of the CNN's output representation when continuously rotating the object
and found that it has to form a closed loop after each step of symmetry.
Otherwise, the CNN (which is itself a continuous function) has to replicate an
uncontinuous function. On a 1-DOF toy example we show that commonly used
representations do not fulfill this demand and analyze the problems caused
thereby. In particular, we find that the popular min-over-symmetries approach
for creating a symmetry-aware loss tends not to work well with gradient-based
optimization, i.e. deep learning.
  We propose a representation called "closed symmetry loop" (csl) from these
insights, where the angle of relevant vectors is multiplied by the symmetry
order and then generalize it to 6-DOF. The representation extends our algorithm
from [Richter-Klug, ICVS, 2019] including a method to disambiguate symmetric
equivalents during the final pose estimation. The algorithm handles continuous
rotational symmetry (e.g. a bottle) and discrete rotational symmetry (e.g. a
4-fold symmetric box). It is evaluated on the T-LESS dataset, where it reaches
state-of-the-art for unrefining RGB-based methods.
