  Human demonstrations of trajectories are an important source of training data
for many machine learning problems. However, the difficulty of collecting human
demonstration data for complex tasks makes learning efficient representations
of those trajectories challenging. For many problems, such as for dexterous
manipulation, the exact timings of the trajectories should be factored from
their spatial path characteristics. In this work, we propose TimewarpVAE, a
fully differentiable manifold-learning algorithm that incorporates Dynamic Time
Warping (DTW) to simultaneously learn both timing variations and latent factors
of spatial variation. We show how the TimewarpVAE algorithm learns appropriate
time alignments and meaningful representations of spatial variations in
handwriting and fork manipulation datasets. Our results have lower spatial
reconstruction test error than baseline approaches and the learned
low-dimensional representations can be used to efficiently generate
semantically meaningful novel trajectories. We demonstrate the utility of our
algorithm to generate novel high-speed trajectories for a robotic arm.
