  We develop deep Poisson-gamma dynamical systems (DPGDS) to model sequentially
observed multivariate count data, improving previously proposed models by not
only mining deep hierarchical latent structure from the data, but also
capturing both first-order and long-range temporal dependencies. Using
sophisticated but simple-to-implement data augmentation techniques, we derived
closed-form Gibbs sampling update equations by first backward and upward
propagating auxiliary latent counts, and then forward and downward sampling
latent variables. Moreover, we develop stochastic gradient MCMC inference that
is scalable to very long multivariate count time series. Experiments on both
synthetic and a variety of real-world data demonstrate that the proposed model
not only has excellent predictive performance, but also provides highly
interpretable multilayer latent structure to represent hierarchical and
temporal information propagation.
