  Distinction among nearby poses and among symmetries of an object is
challenging. In this paper, we propose a unified, group-theoretic approach to
tackle both. Different from existing works which directly predict absolute
pose, our method measures the pose of an object relative to another pose, i.e.,
the pose difference. The proposed method generates the complete orbit of an
object from a single view of the object with respect to the subgroup of SO(3)
of rotations around the z-axis, and compares the orbit of the object with
another orbit using a novel orbit metric to estimate the pose difference. The
generated orbit in the latent space records all the differences in pose in the
original observational space, and as a result, the method is capable of finding
subtle differences in pose. We demonstrate the effectiveness of the proposed
method on cars, where identifying the subtle pose differences is vital.
