  Accurate object rearrangement from vision is a crucial problem for a wide
variety of real-world robotics applications in unstructured environments. We
propose IFOR, Iterative Flow Minimization for Robotic Object Rearrangement, an
end-to-end method for the challenging problem of object rearrangement for
unknown objects given an RGBD image of the original and final scenes. First, we
learn an optical flow model based on RAFT to estimate the relative
transformation of the objects purely from synthetic data. This flow is then
used in an iterative minimization algorithm to achieve accurate positioning of
previously unseen objects. Crucially, we show that our method applies to
cluttered scenes, and in the real world, while training only on synthetic data.
Videos are available at https://imankgoyal.github.io/ifor.html.
