  Variational inference has had great success in scaling approximate Bayesian
inference to big data by exploiting mini-batch training. To date, however, this
strategy has been most applicable to models of independent data. We propose an
extension to state space models of time series data based on a novel generative
model for latent temporal states: the neural moving average model. This permits
a subsequence to be sampled without drawing from the entire distribution,
enabling training iterations to use mini-batches of the time series at low
computational cost. We illustrate our method on autoregressive, Lotka-Volterra,
FitzHugh-Nagumo and stochastic volatility models, achieving accurate parameter
estimation in a short time.
