  Conventional saliency prediction models typically learn a deterministic
mapping from an image to its saliency map, and thus fail to explain the
subjective nature of human attention. In this paper, to model the uncertainty
of visual saliency, we study the saliency prediction problem from the
perspective of generative models by learning a conditional probability
distribution over the saliency map given an input image, and treating the
saliency prediction as a sampling process from the learned distribution.
Specifically, we propose a generative cooperative saliency prediction
framework, where a conditional latent variable model (LVM) and a conditional
energy-based model (EBM) are jointly trained to predict salient objects in a
cooperative manner. The LVM serves as a fast but coarse predictor to
efficiently produce an initial saliency map, which is then refined by the
iterative Langevin revision of the EBM that serves as a slow but fine
predictor. Such a coarse-to-fine cooperative saliency prediction strategy
offers the best of both worlds. Moreover, we propose a "cooperative learning
while recovering" strategy and apply it to weakly supervised saliency
prediction, where saliency annotations of training images are partially
observed. Lastly, we find that the learned energy function in the EBM can serve
as a refinement module that can refine the results of other pre-trained
saliency prediction models. Experimental results show that our model can
produce a set of diverse and plausible saliency maps of an image, and obtain
state-of-the-art performance in both fully supervised and weakly supervised
saliency prediction tasks.
