  Extrapolation remains a grand challenge in deep neural networks across all
application domains. We propose an operator learning method to solve
time-dependent partial differential equations (PDEs) continuously and with
extrapolation in time without any temporal discretization. The proposed method,
named Diffusion-inspired Temporal Transformer Operator (DiTTO), is inspired by
latent diffusion models and their conditioning mechanism, which we use to
incorporate the temporal evolution of the PDE, in combination with elements
from the transformer architecture to improve its capabilities. Upon training,
DiTTO can make inferences in real-time. We demonstrate its extrapolation
capability on a climate problem by estimating the temperature around the globe
for several years, and also in modeling hypersonic flows around a double-cone.
We propose different training strategies involving temporal-bundling and
sub-sampling and demonstrate performance improvements for several benchmarks,
performing extrapolation for long time intervals as well as zero-shot
super-resolution in time.
