  The increasing complexity of tasks in robotics demands efficient strategies
for multitask and continual learning. Traditional models typically rely on a
universal policy for all tasks, facing challenges such as high computational
costs and catastrophic forgetting when learning new tasks. To address these
issues, we introduce a sparse, reusable, and flexible policy, Sparse Diffusion
Policy (SDP). By adopting Mixture of Experts (MoE) within a transformer-based
diffusion policy, SDP selectively activates experts and skills, enabling
efficient and task-specific learning without retraining the entire model. SDP
not only reduces the burden of active parameters but also facilitates the
seamless integration and reuse of experts across various tasks. Extensive
experiments on diverse tasks in both simulations and real world show that SDP
1) excels in multitask scenarios with negligible increases in active
parameters, 2) prevents forgetting in continual learning of new tasks, and 3)
enables efficient task transfer, offering a promising solution for advanced
robotic applications. Demos and codes can be found in
https://forrest-110.github.io/sparse_diffusion_policy/.
