  Diffusion models have shown strong competitiveness in offline reinforcement
learning tasks by formulating decision-making as sequential generation.
However, the practicality of these methods is limited due to the lengthy
inference processes they require. In this paper, we address this problem by
decomposing the sampling process of diffusion models into two decoupled
subprocesses: 1) generating a feasible trajectory, which is a time-consuming
process, and 2) optimizing the trajectory. With this decomposition approach, we
are able to partially separate efficiency and quality factors, enabling us to
simultaneously gain efficiency advantages and ensure quality assurance. We
propose the Trajectory Diffuser, which utilizes a faster autoregressive model
to handle the generation of feasible trajectories while retaining the
trajectory optimization process of diffusion models. This allows us to achieve
more efficient planning without sacrificing capability. To evaluate the
effectiveness and efficiency of the Trajectory Diffuser, we conduct experiments
on the D4RL benchmarks. The results demonstrate that our method achieves $\it
3$-$\it 10 \times$ faster inference speed compared to previous sequence
modeling methods, while also outperforming them in terms of overall
performance. https://github.com/RenMing-Huang/TrajectoryDiffuser
  Keywords: Reinforcement Learning and Efficient Planning and Diffusion Model
