  Anatomical segmentation is a fundamental task in medical image computing,
generally tackled with fully convolutional neural networks which produce dense
segmentation masks. These models are often trained with loss functions such as
cross-entropy or Dice, which assume pixels to be independent of each other,
thus ignoring topological errors and anatomical inconsistencies. We address
this limitation by moving from pixel-level to graph representations, which
allow to naturally incorporate anatomical constraints by construction. To this
end, we introduce HybridGNet, an encoder-decoder neural architecture that
leverages standard convolutions for image feature encoding and graph
convolutional neural networks (GCNNs) to decode plausible representations of
anatomical structures. We also propose a novel image-to-graph skip connection
layer which allows localized features to flow from standard convolutional
blocks to GCNN blocks, and show that it improves segmentation accuracy. The
proposed architecture is extensively evaluated in a variety of domain shift and
image occlusion scenarios, and audited considering different types of
demographic domain shift. Our comprehensive experimental setup compares
HybridGNet with other landmark and pixel-based models for anatomical
segmentation in chest x-ray images, and shows that it produces anatomically
plausible results in challenging scenarios where other models tend to fail.
