  6D pose estimation of textureless objects is a valuable but challenging task
for many robotic applications. In this work, we propose a framework to address
this challenge using only RGB images acquired from multiple viewpoints. The
core idea of our approach is to decouple 6D pose estimation into a sequential
two-step process, first estimating the 3D translation and then the 3D rotation
of each object. This decoupled formulation first resolves the scale and depth
ambiguities in single RGB images, and uses these estimates to accurately
identify the object orientation in the second stage, which is greatly
simplified with an accurate scale estimate. Moreover, to accommodate the
multi-modal distribution present in rotation space, we develop an optimization
scheme that explicitly handles object symmetries and counteracts measurement
uncertainties. In comparison to the state-of-the-art multi-view approach, we
demonstrate that the proposed approach achieves substantial improvements on a
challenging 6D pose estimation dataset for textureless objects.
