  Deep learning stands at the forefront in many computer vision tasks. However,
deep neural networks are usually data-hungry and require a huge amount of
well-annotated training samples. Collecting sufficient annotated data is very
expensive in many applications, especially for pixel-level prediction tasks
such as semantic segmentation. To solve this fundamental issue, we consider a
new challenging vision task, Internetly supervised semantic segmentation, which
only uses Internet data with noisy image-level supervision of corresponding
query keywords for segmentation model training. We address this task by
proposing the following solution. A class-specific attention model unifying
multiscale forward and backward convolutional features is proposed to provide
initial segmentation "ground truth". The model trained with such noisy
annotations is then improved by an online fine-tuning procedure. It achieves
state-of-the-art performance under the weakly-supervised setting on PASCAL
VOC2012 dataset. The proposed framework also paves a new way towards learning
from the Internet without human interaction and could serve as a strong
baseline therein. Code and data will be released upon the paper acceptance.
