  Distribution shifts are problems where the distribution of data changes
between training and testing, which can significantly degrade the performance
of a model deployed in the real world. Recent studies suggest that one reason
for the degradation is a type of overfitting, and that proper regularization
can mitigate the degradation, especially when using highly representative
models such as neural networks. In this paper, we propose a new regularization
using the supervised contrastive learning to prevent such overfitting and to
train models that do not degrade their performance under the distribution
shifts. We extend the cosine similarity in contrastive loss to a more general
similarity measure and propose to use different parameters in the measure when
comparing a sample to a positive or negative example, which is analytically
shown to act as a kind of margin in contrastive loss. Experiments on benchmark
datasets that emulate distribution shifts, including subpopulation shift and
domain generalization, demonstrate the advantage of the proposed method over
existing regularization methods.
