  Human environments contain numerous objects configured in a variety of
arrangements. Our goal is to enable robots to repose previously unseen objects
according to learned semantic relationships in novel environments. We break
this problem down into two parts: (1) finding physically valid locations for
the objects and (2) determining if those poses satisfy learned, high-level
semantic relationships. We build our models and training from the ground up to
be tightly integrated with our proposed planning algorithm for semantic
placement of unknown objects. We train our models purely in simulation, with no
fine-tuning needed for use in the real world. Our approach enables motion
planning for semantic rearrangement of unknown objects in scenes with varying
geometry from only RGB-D sensing. Our experiments through a set of simulated
ablations demonstrate that using a relational classifier alone is not
sufficient for reliable planning. We further demonstrate the ability of our
planner to generate and execute diverse manipulation plans through a set of
real-world experiments with a variety of objects.
