  In this paper, we study the problem of procedure planning in instructional
videos, which aims to make goal-directed plans given the current visual
observations in unstructured real-life videos. Previous works cast this problem
as a sequence planning problem and leverage either heavy intermediate visual
observations or natural language instructions as supervision, resulting in
complex learning schemes and expensive annotation costs. In contrast, we treat
this problem as a distribution fitting problem. In this sense, we model the
whole intermediate action sequence distribution with a diffusion model (PDPP),
and thus transform the planning problem to a sampling process from this
distribution. In addition, we remove the expensive intermediate supervision,
and simply use task labels from instructional videos as supervision instead.
Our model is a U-Net based diffusion model, which directly samples action
sequences from the learned distribution with the given start and end
observations. Furthermore, we apply an efficient projection method to provide
accurate conditional guides for our model during the learning and sampling
process. Experiments on three datasets with different scales show that our PDPP
model can achieve the state-of-the-art performance on multiple metrics, even
without the task supervision. Code and trained models are available at
https://github.com/MCG-NJU/PDPP.
