  We propose a theoretical framework for studying behavior cloning of complex
expert demonstrations using generative modeling. Our framework invokes
low-level controllers - either learned or implicit in position-command control
- to stabilize imitation around expert demonstrations. We show that with (a) a
suitable low-level stability guarantee and (b) a powerful enough generative
model as our imitation learner, pure supervised behavior cloning can generate
trajectories matching the per-time step distribution of essentially arbitrary
expert trajectories in an optimal transport cost. Our analysis relies on a
stochastic continuity property of the learned policy we call "total variation
continuity" (TVC). We then show that TVC can be ensured with minimal
degradation of accuracy by combining a popular data-augmentation regimen with a
novel algorithmic trick: adding augmentation noise at execution time. We
instantiate our guarantees for policies parameterized by diffusion models and
prove that if the learner accurately estimates the score of the
(noise-augmented) expert policy, then the distribution of imitator trajectories
is close to the demonstrator distribution in a natural optimal transport
distance. Our analysis constructs intricate couplings between noise-augmented
trajectories, a technique that may be of independent interest. We conclude by
empirically validating our algorithmic recommendations, and discussing
implications for future research directions for better behavior cloning with
generative modeling.
