  Diffusion models have risen as a promising approach to data-driven planning,
and have demonstrated impressive robotic control, reinforcement learning, and
video planning performance. Given an effective planner, an important question
to consider is replanning -- when given plans should be regenerated due to both
action execution error and external environment changes. Direct plan execution,
without replanning, is problematic as errors from individual actions rapidly
accumulate and environments are partially observable and stochastic.
Simultaneously, replanning at each timestep incurs a substantial computational
cost, and may prevent successful task execution, as different generated plans
prevent consistent progress to any particular goal. In this paper, we explore
how we may effectively replan with diffusion models. We propose a principled
approach to determine when to replan, based on the diffusion model's estimated
likelihood of existing generated plans. We further present an approach to
replan existing trajectories to ensure that new plans follow the same goal
state as the original trajectory, which may efficiently bootstrap off
previously generated plans. We illustrate how a combination of our proposed
additions significantly improves the performance of diffusion planners leading
to 38\% gains over past diffusion planning approaches on Maze2D, and further
enables the handling of stochastic and long-horizon robotic control tasks.
Videos can be found on the anonymous website:
\url{https://vis-www.cs.umass.edu/replandiffuser/}.
