  Deep learning based pipelines for semantic segmentation often ignore
structural information available on annotated images used for training. We
propose a novel post-processing module enforcing structural knowledge about the
objects of interest to improve segmentation results provided by deep learning.
This module corresponds to a "many-to-one-or-none" inexact graph matching
approach, and is formulated as a quadratic assignment problem. Our approach is
compared to a CNN-based segmentation (for various CNN backbones) on two public
datasets, one for face segmentation from 2D RGB images (FASSEG), and the other
for brain segmentation from 3D MRIs (IBSR). Evaluations are performed using two
types of structural information (distances and directional relations, , this
choice being a hyper-parameter of our generic framework). On FASSEG data,
results show that our module improves accuracy of the CNN by about 6.3% (the
Hausdorff distance decreases from 22.11 to 20.71). On IBSR data, the
improvement is of 51% (the Hausdorff distance decreases from 11.01 to 5.4). In
addition, our approach is shown to be resilient to small training datasets that
often limit the performance of deep learning methods: the improvement increases
as the size of the training dataset decreases.
