  This paper investigates the initialization bias of the Fourier neural
operator (FNO). A mean-field theory for FNO is established, analyzing the
behavior of the random FNO from an \emph{edge of chaos} perspective. We uncover
that the forward and backward propagation behaviors exhibit characteristics
unique to FNO, induced by mode truncation, while also showcasing similarities
to those of densely connected networks. Building upon this observation, we also
propose an edge of chaos initialization scheme for FNO to mitigate the negative
initialization bias leading to training instability. Experimental results show
the effectiveness of our initialization scheme, enabling stable training of
deep FNO without skip-connection.
