  Variational inference algorithms have proven successful for Bayesian analysis
in large data settings, with recent advances using stochastic variational
inference (SVI). However, such methods have largely been studied in independent
or exchangeable data settings. We develop an SVI algorithm to learn the
parameters of hidden Markov models (HMMs) in a time-dependent data setting. The
challenge in applying stochastic optimization in this setting arises from
dependencies in the chain, which must be broken to consider minibatches of
observations. We propose an algorithm that harnesses the memory decay of the
chain to adaptively bound errors arising from edge effects. We demonstrate the
effectiveness of our algorithm on synthetic experiments and a large genomics
dataset where a batch algorithm is computationally infeasible.
