  We study the quadratic prediction error method -- i.e., nonlinear least
squares -- for a class of time-varying parametric predictor models satisfying a
certain identifiability condition. While this method is known to asymptotically
achieve the optimal rate for a wide range of problems, there have been no
non-asymptotic results matching these optimal rates outside of a select few,
typically linear, model classes. By leveraging modern tools from learning with
dependent data, we provide the first rate-optimal non-asymptotic analysis of
this method for our more general setting of nonlinearly parametrized model
classes. Moreover, we show that our results can be applied to a particular
class of identifiable AutoRegressive Moving Average (ARMA) models, resulting in
the first optimal non-asymptotic rates for identification of ARMA models.
