  We introduce and study a variational framework for the analysis of empirical
risk based inference for dynamical systems and ergodic processes. The analysis
applies to a two-stage estimation procedure in which (i) the trajectory of an
observed (but unknown) system is fit to a trajectory from a known reference
system by minimizing cumulative per-state loss, and (ii) a parameter estimate
is obtained from the initial state of the best fit reference trajectory. We
show that the empirical risk of the best fit trajectory converges almost surely
to a constant that can be expressed in a variational form as the minimal
expected loss over dynamically invariant couplings (joinings) of the observed
and reference systems. Moreover, we establish that the family of joinings
minimizing the expected loss is convex and compact, and that it fully
characterizes the asymptotic behavior of the estimated parameters, addressing
both identifiability and misspecification. The two-stage estimation framework
and associated variational analysis apply to a broad family of empirical risk
miminization procedures for dependent observations. To illustrate this, we
apply variational analysis to the well studied problems of maximum likelihood
and non-linear regression, and then undertake an extended analysis of system
identification from quantized trajectories subject to noise, a problem of
interest in dynamics, where the models themselves exhibit dynamical behavior
across time.
