  While class activation map (CAM) generated by image classification network
has been widely used for weakly supervised object localization (WSOL) and
semantic segmentation (WSSS), such classifiers usually focus on discriminative
object regions. In this paper, we propose Contrastive learning for
Class-agnostic Activation Map (C$^2$AM) generation only using unlabeled image
data, without the involvement of image-level supervision. The core idea comes
from the observation that i) semantic information of foreground objects usually
differs from their backgrounds; ii) foreground objects with similar appearance
or background with similar color/texture have similar representations in the
feature space. We form the positive and negative pairs based on the above
relations and force the network to disentangle foreground and background with a
class-agnostic activation map using a novel contrastive loss. As the network is
guided to discriminate cross-image foreground-background, the class-agnostic
activation maps learned by our approach generate more complete object regions.
We successfully extracted from C$^2$AM class-agnostic object bounding boxes for
object localization and background cues to refine CAM generated by
classification network for semantic segmentation. Extensive experiments on
CUB-200-2011, ImageNet-1K, and PASCAL VOC2012 datasets show that both WSOL and
WSSS can benefit from the proposed C$^2$AM.
