  This paper establishes optimal convergence rates for estimation of structured
covariance operators of Gaussian processes. We study banded operators with
kernels that decay rapidly off-the-diagonal and $L^q$-sparse operators with an
unordered sparsity pattern. For these classes of operators, we find the minimax
optimal rate of estimation in operator norm, identifying the fundamental
dimension-free quantities that determine the sample complexity. In addition, we
prove that tapering and thresholding estimators attain the optimal rate. The
proof of the upper bound for tapering estimators requires novel techniques to
circumvent the issue that discretization of a banded operator does not result,
in general, in a banded covariance matrix. To derive lower bounds for banded
and $L^q$-sparse classes, we introduce a general framework to lift theory from
high-dimensional matrix estimation to the operator setting. Our work
contributes to the growing literature on operator estimation and learning,
building on ideas from high-dimensional statistics while also addressing new
challenges that emerge in infinite dimension.
