  6-DoF pose estimation is an essential component of robotic manipulation
pipelines. However, it usually suffers from a lack of generalization to new
instances and object types. Most widely used methods learn to infer the object
pose in a discriminative setup where the model filters useful information to
infer the exact pose of the object. While such methods offer accurate poses,
the model does not store enough information to generalize to new objects. In
this work, we address the generalization capability of pose estimation using
models that contain enough information about the object to render it in
different poses. We follow the line of work that inverts neural renderers to
infer the pose. We propose i-$\sigma$SRN to maximize the information flowing
from the input pose to the rendered scene and invert them to infer the pose
given an input image. Specifically, we extend Scene Representation Networks
(SRNs) by incorporating a separate network for density estimation and introduce
a new way of obtaining a weighted scene representation. We investigate several
ways of initial pose estimates and losses for the neural renderer. Our final
evaluation shows a significant improvement in inference performance and speed
compared to existing approaches.
