  The notion of memory capacity, originally introduced for echo state and
linear networks with independent inputs, is generalized to nonlinear recurrent
networks with stationary but dependent inputs. The presence of dependence in
the inputs makes natural the introduction of the network forecasting capacity,
that measures the possibility of forecasting time series values using network
states. Generic bounds for memory and forecasting capacities are formulated in
terms of the number of neurons of the nonlinear recurrent network and the
autocovariance function or the spectral density of the input. These bounds
generalize well-known estimates in the literature to a dependent inputs setup.
Finally, for the particular case of linear recurrent networks with independent
inputs it is proved that the memory capacity is given by the rank of the
associated controllability matrix, a fact that has been for a long time assumed
to be true without proof by the community.
