  We characterize the class of image plane transformations which realize rigid
camera motions and call these transformations `rigidity preserving'. In
particular, 2D translations of pinhole images are not rigidity preserving.
Hence, when using CNNs for 3D inference tasks, it can be beneficial to modify
the inductive bias from equivariance towards translations to equivariance
towards rigidity preserving transformations. We investigate how equivariance
with respect to rigidity preserving transformations can be approximated in
CNNs, and test our ideas on both 6D object pose estimation and visual
localization. Experimentally, we improve on several competitive baselines.
