  In this work we introduce a new Bounding-Box Free Network (BBFNet) for
panoptic segmentation. Panoptic segmentation is an ideal problem for
proposal-free methods as it already requires per-pixel semantic class labels.
We use this observation to exploit class boundaries from off-the-shelf semantic
segmentation networks and refine them to predict instance labels. Towards this
goal BBFNet predicts coarse watershed levels and uses them to detect large
instance candidates where boundaries are well defined. For smaller instances,
whose boundaries are less reliable, BBFNet also predicts instance centers by
means of Hough voting followed by mean-shift to reliably detect small objects.
A novel triplet loss network helps merging fragmented instances while refining
boundary pixels. Our approach is distinct from previous works in panoptic
segmentation that rely on a combination of a semantic segmentation network with
a computationally costly instance segmentation network based on bounding box
proposals, such as Mask R-CNN, to guide the prediction of instance labels using
a Mixture-of-Expert (MoE) approach. We benchmark our proposal-free method on
Cityscapes and Microsoft COCO datasets and show competitive performance with
other MoE based approaches while outperforming existing non-proposal based
methods on the COCO dataset. We show the flexibility of our method using
different semantic segmentation backbones.
