  An important problem in time-series analysis is modeling systems with
time-varying dynamics. Probabilistic models with joint continuous and discrete
latent states offer interpretable, efficient, and experimentally useful
descriptions of such data. Commonly used models include autoregressive hidden
Markov models (ARHMMs) and switching linear dynamical systems (SLDSs), each
with its own advantages and disadvantages. ARHMMs permit exact inference and
easy parameter estimation, but are parameter intensive when modeling long
dependencies, and hence are prone to overfitting. In contrast, SLDSs can
capture long-range dependencies in a parameter efficient way through Markovian
latent dynamics, but present an intractable likelihood and a challenging
parameter estimation task. In this paper, we propose switching autoregressive
low-rank tensor (SALT) models, which retain the advantages of both approaches
while ameliorating the weaknesses. SALT parameterizes the tensor of an ARHMM
with a low-rank factorization to control the number of parameters and allow
longer range dependencies without overfitting. We prove theoretical and discuss
practical connections between SALT, linear dynamical systems, and SLDSs. We
empirically demonstrate quantitative advantages of SALT models on a range of
simulated and real prediction tasks, including behavioral and neural datasets.
Furthermore, the learned low-rank tensor provides novel insights into temporal
dependencies within each discrete state.
