  Existing weakly or semi-supervised semantic segmentation methods utilize
image or box-level supervision to generate pseudo-labels for weakly labeled
images. However, due to the lack of strong supervision, the generated
pseudo-labels are often noisy near the object boundaries, which severely
impacts the network's ability to learn strong representations. To address this
problem, we present a novel framework that generates pseudo-labels for training
images, which are then used to train a segmentation model. To generate
pseudo-labels, we combine information from: (i) a class agnostic objectness
network that learns to recognize object-like regions, and (ii) either
image-level or bounding box annotations. We show the efficacy of our approach
by demonstrating how the objectness network can naturally be leveraged to
generate object-like regions for unseen categories. We then propose an
end-to-end multi-task learning strategy, that jointly learns to segment
semantics and objectness using the generated pseudo-labels. Extensive
experiments demonstrate the high quality of our generated pseudo-labels and
effectiveness of the proposed framework in a variety of domains. Our approach
achieves better or competitive performance compared to existing
weakly-supervised and semi-supervised methods.
