  Covariance estimation is ubiquitous in functional data analysis. Yet, the
case of functional observations over multidimensional domains introduces
computational and statistical challenges, rendering the standard methods
effectively inapplicable. To address this problem, we introduce "Covariance
Networks" (CovNet) as a modeling and estimation tool. The CovNet model is
"universal" - it can be used to approximate any covariance up to desired
precision. Moreover, the model can be fitted efficiently to the data and its
neural network architecture allows us to employ modern computational tools in
the implementation. The CovNet model also admits a closed-form
eigendecomposition, which can be computed efficiently, without constructing the
covariance itself. This facilitates easy storage and subsequent manipulation of
a covariance in the context of the CovNet. We establish consistency of the
proposed estimator and derive its rate of convergence. The usefulness of the
proposed method is demonstrated by means of an extensive simulation study and
an application to resting state fMRI data.
