  We present a two-module approach to semantic segmentation that incorporates
Convolutional Networks (CNNs) and Graphical Models. Graphical models are used
to generate a small (5-30) set of diverse segmentations proposals, such that
this set has high recall. Since the number of required proposals is so low, we
can extract fairly complex features to rank them. Our complex feature of choice
is a novel CNN called SegNet, which directly outputs a (coarse) semantic
segmentation. Importantly, SegNet is specifically trained to optimize the
corpus-level PASCAL IOU loss function. To the best of our knowledge, this is
the first CNN specifically designed for semantic segmentation. This two-module
approach achieves $52.5\%$ on the PASCAL 2012 segmentation challenge.
