  Mastering dexterous robotic manipulation of deformable objects is vital for
overcoming the limitations of parallel grippers in real-world applications.
Current trajectory optimisation approaches often struggle to solve such tasks
due to the large search space and the limited task information available from a
cost function. In this work, we propose D-Cubed, a novel trajectory
optimisation method using a latent diffusion model (LDM) trained from a
task-agnostic play dataset to solve dexterous deformable object manipulation
tasks. D-Cubed learns a skill-latent space that encodes short-horizon actions
in the play dataset using a VAE and trains a LDM to compose the skill latents
into a skill trajectory, representing a long-horizon action trajectory in the
dataset. To optimise a trajectory for a target task, we introduce a novel
gradient-free guided sampling method that employs the Cross-Entropy method
within the reverse diffusion process. In particular, D-Cubed samples a small
number of noisy skill trajectories using the LDM for exploration and evaluates
the trajectories in simulation. Then, D-Cubed selects the trajectory with the
lowest cost for the subsequent reverse process. This effectively explores
promising solution areas and optimises the sampled trajectories towards a
target task throughout the reverse diffusion process. Through empirical
evaluation on a public benchmark of dexterous deformable object manipulation
tasks, we demonstrate that D-Cubed outperforms traditional trajectory
optimisation and competitive baseline approaches by a significant margin. We
further demonstrate that trajectories found by D-Cubed readily transfer to a
real-world LEAP hand on a folding task.
