  Invariance learning methods aim to learn invariant features in the hope that
they generalize under distributional shifts. Although many tasks are naturally
characterized by continuous domains, current invariance learning techniques
generally assume categorically indexed domains. For example, auto-scaling in
cloud computing often needs a CPU utilization prediction model that generalizes
across different times (e.g., time of a day and date of a year), where `time'
is a continuous domain index. In this paper, we start by theoretically showing
that existing invariance learning methods can fail for continuous domain
problems. Specifically, the naive solution of splitting continuous domains into
discrete ones ignores the underlying relationship among domains, and therefore
potentially leads to suboptimal performance. To address this challenge, we then
propose Continuous Invariance Learning (CIL), which extracts invariant features
across continuously indexed domains. CIL is a novel adversarial procedure that
measures and controls the conditional independence between the labels and
continuous domain indices given the extracted features. Our theoretical
analysis demonstrates the superiority of CIL over existing invariance learning
methods. Empirical results on both synthetic and real-world datasets (including
data collected from production systems) show that CIL consistently outperforms
strong baselines among all the tasks.
