  The deployment of machine learning models in safety-critical applications
comes with the expectation that such models will perform well over a range of
contexts (e.g., a vision model for classifying street signs should work in
rural, city, and highway settings under varying lighting/weather conditions).
However, these one-size-fits-all models are typically optimized for average
case performance, encouraging them to achieve high performance in nominal
conditions but exposing them to unexpected behavior in challenging or rare
contexts. To address this concern, we develop a new method for training
context-dependent models. We extend Bridge-Mode Connectivity (BMC) (Garipov et
al., 2018) to train an infinite ensemble of models over a continuous measure of
context such that we can sample model parameters specifically tuned to the
corresponding evaluation context. We explore the definition of context in image
classification tasks through multiple lenses including changes in the risk
profile, long-tail image statistics/appearance, and context-dependent
distribution shift. We develop novel extensions of the BMC optimization for
each of these cases and our experiments demonstrate that model performance can
be successfully tuned to context in each scenario.
