  Robotic systems often require precise scene analysis capabilities, especially
in unstructured, cluttered situations, as occurring in human-made environments.
While current deep-learning based methods yield good estimates of object poses,
they often struggle with large amounts of occlusion and do not take
inter-object effects into account. Vision as inverse graphics is a promising
concept for detailed scene analysis. A key element for this idea is a method
for inferring scene parameter updates from the rasterized 2D scene. However,
the rasterization process is notoriously difficult to invert, both due to the
projection and occlusion process, but also due to secondary effects such as
lighting or reflections. We propose to remove the latter from the process by
mapping the rasterized image into an abstract feature space learned in a
self-supervised way from pixel correspondences. Using only a light-weight
inverse rendering module, this allows us to refine 6D object pose estimations
in highly cluttered scenes by optimizing a simple pixel-wise difference in the
abstract image representation. We evaluate our approach on the challenging
YCB-Video dataset, where it yields large improvements and demonstrates a large
basin of attraction towards the correct object poses.
