  Operator learning based on neural operators has emerged as a promising
paradigm for the data-driven approximation of operators, mapping between
infinite-dimensional Banach spaces. Despite significant empirical progress, our
theoretical understanding regarding the efficiency of these approximations
remains incomplete. This work addresses the parametric complexity of neural
operator approximations for the general class of Lipschitz continuous
operators. Motivated by recent findings on the limitations of specific
architectures, termed curse of parametric complexity, we here adopt an
information-theoretic perspective. Our main contribution establishes lower
bounds on the metric entropy of Lipschitz operators in two approximation
settings; uniform approximation over a compact set of input functions, and
approximation in expectation, with input functions drawn from a probability
measure. It is shown that these entropy bounds imply that, regardless of the
activation function used, neural operator architectures attaining an
approximation accuracy $\epsilon$ must have a size that is exponentially large
in $\epsilon^{-1}$. The size of architectures is here measured by counting the
number of encoded bits necessary to store the given model in computational
memory. The results of this work elucidate fundamental trade-offs and
limitations in operator learning.
