  This paper details how the Bayesian-network structure of the posterior
distribution of state-space models can be exploited to build improved
parameterizations for system identification using variational inference. Three
different parameterizations of the assumed state-path posterior distribution
are proposed based on this representation: time-varying, steady-state, and
convolution-smoother; each resulting in a different parameter estimation
method. In contrast to existing methods for variational system identification,
the proposed estimators can be implemented with unconstrained optimization
methods. Furthermore, when applied to mini-batches in conjunction with
stochastic optimization methods, the convolution-smoother formulation enables
identification of large linear and nonlinear state-space systems from very
large datasets. For linear systems, the method achieves the same performance as
the inherently sequential prediction-error methods using and embarrassingly
parallel algorithm that benefits from large speedups when computed in modern
graphical processing units (GPUs). The ability of the proposed estimators to
identify large models, work with large datasets split into mini-batches, and be
work in parallel on GPUs make them well-suited for identifying deep models for
applications in systems and control.
