  Weakly-supervised semantic segmentation (WSSS) has recently gained much
attention for its promise to train segmentation models only with image-level
labels. Existing WSSS methods commonly argue that the sparse coverage of CAM
incurs the performance bottleneck of WSSS. This paper provides analytical and
empirical evidence that the actual bottleneck may not be sparse coverage but a
global thresholding scheme applied after CAM. Then, we show that this issue can
be mitigated by satisfying two conditions; 1) reducing the imbalance in the
foreground activation and 2) increasing the gap between the foreground and the
background activation. Based on these findings, we propose a novel activation
manipulation network with a per-pixel classification loss and a label
conditioning module. Per-pixel classification naturally induces two-level
activation in activation maps, which can penalize the most discriminative
parts, promote the less discriminative parts, and deactivate the background
regions. Label conditioning imposes that the output label of pseudo-masks
should be any of true image-level labels; it penalizes the wrong activation
assigned to non-target classes. Based on extensive analysis and evaluations, we
demonstrate that each component helps produce accurate pseudo-masks, achieving
the robustness against the choice of the global threshold. Finally, our model
achieves state-of-the-art records on both PASCAL VOC 2012 and MS COCO 2014
datasets.
