  Recent years have witnessed the promise of coupling machine learning methods
and physical domainspecific insights for solving scientific problems based on
partial differential equations (PDEs). However, being data-intensive, these
methods still require a large amount of PDE data. This reintroduces the need
for expensive numerical PDE solutions, partially undermining the original goal
of avoiding these expensive simulations. In this work, seeking data efficiency,
we design unsupervised pretraining for PDE operator learning. To reduce the
need for training data with heavy simulation costs, we mine unlabeled PDE data
without simulated solutions, and pretrain neural operators with
physics-inspired reconstruction-based proxy tasks. To improve
out-of-distribution performance, we further assist neural operators in flexibly
leveraging in-context learning methods, without incurring extra training costs
or designs. Extensive empirical evaluations on a diverse set of PDEs
demonstrate that our method is highly data-efficient, more generalizable, and
even outperforms conventional vision-pretrained models.
