  Many biochemical applications such as molecular property prediction require
models to generalize beyond their training domains (environments). Moreover,
natural environments in these tasks are structured, defined by complex
descriptors such as molecular scaffolds or protein families. Therefore, most
environments are either never seen during training, or contain only a single
training example. To address these challenges, we propose a new regret
minimization (RGM) algorithm and its extension for structured environments. RGM
builds from invariant risk minimization (IRM) by recasting simultaneous
optimality condition in terms of predictive regret, finding a representation
that enables the predictor to compete against an oracle with hindsight access
to held-out environments. The structured extension adaptively highlights
variation due to complex environments via specialized domain perturbations. We
evaluate our method on multiple applications: molecular property prediction,
protein homology and stability prediction and show that RGM significantly
outperforms previous state-of-the-art baselines.
