  We propose to model complex visual scenes using a non-parametric Bayesian
model learned from weakly labelled images abundant on media sharing sites such
as Flickr. Given weak image-level annotations of objects and attributes without
locations or associations between them, our model aims to learn the appearance
of object and attribute classes as well as their association on each object
instance. Once learned, given an image, our model can be deployed to tackle a
number of vision problems in a joint and coherent manner, including recognising
objects in the scene (automatic object annotation), describing objects using
their attributes (attribute prediction and association), and localising and
delineating the objects (object detection and semantic segmentation). This is
achieved by developing a novel Weakly Supervised Markov Random Field Stacked
Indian Buffet Process (WS-MRF-SIBP) that models objects and attributes as
latent factors and explicitly captures their correlations within and across
superpixels. Extensive experiments on benchmark datasets demonstrate that our
weakly supervised model significantly outperforms weakly supervised
alternatives and is often comparable with existing strongly supervised models
on a variety of tasks including semantic segmentation, automatic image
annotation and retrieval based on object-attribute associations.
