  Real-world time series data that commonly reflect sequential human behavior
are often uniquely irregularly sampled and sparse, with highly nonuniform
sampling over time and entities. Yet, commonly-used pretraining and
augmentation methods for time series are not specifically designed for such
scenarios. In this paper, we present PAITS (Pretraining and Augmentation for
Irregularly-sampled Time Series), a framework for identifying suitable
pretraining strategies for sparse and irregularly sampled time series datasets.
PAITS leverages a novel combination of NLP-inspired pretraining tasks and
augmentations, and a random search to identify an effective strategy for a
given dataset. We demonstrate that different datasets benefit from different
pretraining choices. Compared with prior methods, our approach is better able
to consistently improve pretraining across multiple datasets and domains. Our
code is available at
\url{https://github.com/google-research/google-research/tree/master/irregular_timeseries_pretraining}.
