  New technologies for recording the activity of large neural populations
during complex behavior provide exciting opportunities for investigating the
neural computations that underlie perception, cognition, and decision-making.
Nonlinear state space models provide an interpretable signal processing
framework by combining an intuitive dynamical system with a probabilistic
observation model, which can provide insights into neural dynamics, neural
computation, and development of neural prosthetics and treatment through
feedback control. It brings the challenge of learning both latent neural state
and the underlying dynamical system because neither is known for neural systems
a priori. We developed a flexible online learning framework for latent
nonlinear state dynamics and filtered latent states. Using the stochastic
gradient variational Bayes approach, our method jointly optimizes the
parameters of the nonlinear dynamical system, the observation model, and the
black-box recognition model. Unlike previous approaches, our framework can
incorporate non-trivial distributions of observation noise and has constant
time and space complexity. These features make our approach amenable to
real-time applications and the potential to automate analysis and experimental
design in ways that testably track and modify behavior using stimuli designed
to influence learning.
