  In this work, we bridge the gap between recent pose estimation and tracking
work to develop a powerful method for robots to track objects in their
surroundings. Motion-Nets use a segmentation model to segment the scene, and
separate translation and rotation models to identify the relative 6D motion of
an object between two consecutive frames. We train our method with generated
data of floating objects, and then test on several prediction tasks, including
one with a real PR2 robot, and a toy control task with a simulated PR2 robot
never seen during training. Motion-Nets are able to track the pose of objects
with some quantitative accuracy for about 30-60 frames including occlusions and
distractors. Additionally, the single step prediction errors remain low even
after 100 frames. We also investigate an iterative correction procedure to
improve performance for control tasks.
