  Diffusion policies have demonstrated robust performance in generative
modeling, prompting their application in robotic manipulation controlled via
language descriptions. In this paper, we introduce a zero-shot, open-vocabulary
diffusion policy method for robot manipulation. Using Vision-Language Models
(VLMs), our method transforms linguistic task descriptions into actionable
keyframes in 3D space. These keyframes serve to guide the diffusion process via
inpainting. However, naively enforcing the diffusion process to adhere to the
generated keyframes is problematic: the keyframes from the VLMs may be
incorrect and lead to out-of-distribution (OOD) action sequences where the
diffusion model performs poorly. To address these challenges, we develop an
inpainting optimization strategy that balances adherence to the keyframes v.s.
the training data distribution. Experimental evaluations demonstrate that our
approach surpasses the performance of traditional fine-tuned
language-conditioned methods in both simulated and real-world settings.
