  Learning nonlinear dynamics from diffusion data is a challenging problem
since the individuals observed may be different at different time points,
generally following an aggregate behaviour. Existing work cannot handle the
tasks well since they model such dynamics either directly on observations or
enforce the availability of complete longitudinal individual-level
trajectories. However, in most of the practical applications, these
requirements are unrealistic: the evolving dynamics may be too complex to be
modeled directly on observations, and individual-level trajectories may not be
available due to technical limitations, experimental costs and/or privacy
issues. To address these challenges, we formulate a model of diffusion dynamics
as the {\em hidden stochastic process} via the introduction of hidden variables
for flexibility, and learn the hidden dynamics directly on {\em aggregate
observations} without any requirement for individual-level trajectories. We
propose a dynamic generative model with Wasserstein distance for LEarninG dEep
hidden Nonlinear Dynamics (LEGEND) and prove its theoretical guarantees as
well. Experiments on a range of synthetic and real-world datasets illustrate
that LEGEND has very strong performance compared to state-of-the-art baselines.
