  Domain-specific variants of contrastive learning can construct positive pairs
from two distinct in-domain images, while traditional methods just augment the
same image twice. For example, we can form a positive pair from two satellite
images showing the same location at different times. Ideally, this teaches the
model to ignore changes caused by seasons, weather conditions or image
acquisition artifacts. However, unlike in traditional contrastive methods, this
can result in undesired positive pairs, since we form them without human
supervision. For example, a positive pair might consist of one image before a
disaster and one after. This could teach the model to ignore the differences
between intact and damaged buildings, which might be what we want to detect in
the downstream task. Similar to false negative pairs, this could impede model
performance. Crucially, in this setting only parts of the images differ in
relevant ways, while other parts remain similar. Surprisingly, we find that
downstream semantic segmentation is either robust to such badly matched pairs
or even benefits from them. The experiments are conducted on the remote sensing
dataset xBD, and a synthetic segmentation dataset for which we have full
control over the pairing conditions. As a result, practitioners can use these
domain-specific contrastive methods without having to filter their positive
pairs beforehand, or might even be encouraged to purposefully include such
pairs in their pretraining dataset.
