  We propose Seg&Struct, a supervised learning framework leveraging the
interplay between part segmentation and structure inference and demonstrating
their synergy in an integrated framework. Both part segmentation and structure
inference have been extensively studied in the recent deep learning literature,
while the supervisions used for each task have not been fully exploited to
assist the other task. Namely, structure inference has been typically conducted
with an autoencoder that does not leverage the point-to-part associations.
Also, segmentation has been mostly performed without structural priors that
tell the plausibility of the output segments. We present how these two tasks
can be best combined while fully utilizing supervision to improve performance.
Our framework first decomposes a raw input shape into part segments using an
off-the-shelf algorithm, whose outputs are then mapped to nodes in a part
hierarchy, establishing point-to-part associations. Following this, ours
predicts the structural information, e.g., part bounding boxes and part
relationships. Lastly, the segmentation is rectified by examining the confusion
of part boundaries using the structure-based part features. Our experimental
results based on the StructureNet and PartNet demonstrate that the interplay
between the two tasks results in remarkable improvements in both tasks: 27.91%
in structure inference and 0.5% in segmentation.
