  Modern datasets in neuroscience enable unprecedented inquiries into the
relationship between complex behaviors and the activity of many simultaneously
recorded neurons. While latent variable models can successfully extract
low-dimensional embeddings from such recordings, using them to generate
realistic spiking data, especially in a behavior-dependent manner, still poses
a challenge. Here, we present Latent Diffusion for Neural Spiking data (LDNS),
a diffusion-based generative model with a low-dimensional latent space: LDNS
employs an autoencoder with structured state-space (S4) layers to project
discrete high-dimensional spiking data into continuous time-aligned latents. On
these inferred latents, we train expressive (conditional) diffusion models,
enabling us to sample neural activity with realistic single-neuron and
population spiking statistics. We validate LDNS on synthetic data, accurately
recovering latent structure, firing rates, and spiking statistics. Next, we
demonstrate its flexibility by generating variable-length data that mimics
human cortical activity during attempted speech. We show how to equip LDNS with
an expressive observation model that accounts for single-neuron dynamics not
mediated by the latent state, further increasing the realism of generated
samples. Finally, conditional LDNS trained on motor cortical activity during
diverse reaching behaviors can generate realistic spiking data given reach
direction or unseen reach trajectories. In summary, LDNS simultaneously enables
inference of low-dimensional latents and realistic conditional generation of
neural spiking datasets, opening up further possibilities for simulating
experimentally testable hypotheses.
