  Weakly supervised object localization remains challenging, where only image
labels instead of bounding boxes are available during training. Object proposal
is an effective component in localization, but often computationally expensive
and incapable of joint optimization with some of the remaining modules. In this
paper, to the best of our knowledge, we for the first time integrate weakly
supervised object proposal into convolutional neural networks (CNNs) in an
end-to-end learning manner. We design a network component, Soft Proposal (SP),
to be plugged into any standard convolutional architecture to introduce the
nearly cost-free object proposal, orders of magnitude faster than
state-of-the-art methods. In the SP-augmented CNNs, referred to as Soft
Proposal Networks (SPNs), iteratively evolved object proposals are generated
based on the deep feature maps then projected back, and further jointly
optimized with network parameters, with image-level supervision only. Through
the unified learning process, SPNs learn better object-centric filters,
discover more discriminative visual evidence, and suppress background
interference, significantly boosting both weakly supervised object localization
and classification performance. We report the best results on popular
benchmarks, including PASCAL VOC, MS COCO, and ImageNet.
