  Do object part localization methods produce bilaterally symmetric results on
mirror images? Surprisingly not, even though state of the art methods augment
the training set with mirrored images. In this paper we take a closer look into
this issue. We first introduce the concept of mirrorability as the ability of a
model to produce symmetric results in mirrored images and introduce a
corresponding measure, namely the \textit{mirror error} that is defined as the
difference between the detection result on an image and the mirror of the
detection result on its mirror image. We evaluate the mirrorability of several
state of the art algorithms in two of the most intensively studied problems,
namely human pose estimation and face alignment. Our experiments lead to
several interesting findings: 1) Surprisingly, most of state of the art methods
struggle to preserve the mirror symmetry, despite the fact that they do have
very similar overall performance on the original and mirror images; 2) the low
mirrorability is not caused by training or testing sample bias - all algorithms
are trained on both the original images and their mirrored versions; 3) the
mirror error is strongly correlated to the localization/alignment error (with
correlation coefficients around 0.7). Since the mirror error is calculated
without knowledge of the ground truth, we show two interesting applications -
in the first it is used to guide the selection of difficult samples and in the
second to give feedback in a popular Cascaded Pose Regression method for face
alignment.
