  Humans can imagine goal states during planning and perform actions to match
those goals. In this work, we propose Imagination Policy, a novel multi-task
key-frame policy network for solving high-precision pick and place tasks.
Instead of learning actions directly, Imagination Policy generates point clouds
to imagine desired states which are then translated to actions using rigid
action estimation. This transforms action inference into a local generative
task. We leverage pick and place symmetries underlying the tasks in the
generation process and achieve extremely high sample efficiency and
generalizability to unseen configurations. Finally, we demonstrate
state-of-the-art performance across various tasks on the RLbench benchmark
compared with several strong baselines.
