  Granular sound synthesis is a popular audio generation technique based on
rearranging sequences of small waveform windows. In order to control the
synthesis, all grains in a given corpus are analyzed through a set of acoustic
descriptors. This provides a representation reflecting some form of local
similarities across the grains. However, the quality of this grain space is
bound by that of the descriptors. Its traversal is not continuously invertible
to signal and does not render any structured temporality.
  We demonstrate that generative neural networks can implement granular
synthesis while alleviating most of its shortcomings. We efficiently replace
its audio descriptor basis by a probabilistic latent space learned with a
Variational Auto-Encoder. In this setting the learned grain space is
invertible, meaning that we can continuously synthesize sound when traversing
its dimensions. It also implies that original grains are not stored for
synthesis. Another major advantage of our approach is to learn structured paths
inside this latent space by training a higher-level temporal embedding over
arranged grain sequences.
  The model can be applied to many types of libraries, including pitched notes
or unpitched drums and environmental noises. We report experiments on the
common granular synthesis processes as well as novel ones such as conditional
sampling and morphing.
