  The pancreatic disease taxonomy includes ten types of masses (tumors or
cysts)[20,8]. Previous work focuses on developing segmentation or
classification methods only for certain mass types. Differential diagnosis of
all mass types is clinically highly desirable [20] but has not been
investigated using an automated image understanding approach. We exploit the
feasibility to distinguish pancreatic ductal adenocarcinoma (PDAC) from the
nine other nonPDAC masses using multi-phase CT imaging. Both image appearance
and the 3D organ-mass geometry relationship are critical. We propose a holistic
segmentation-mesh-classification network (SMCN) to provide patient-level
diagnosis, by fully utilizing the geometry and location information, which is
accomplished by combining the anatomical structure and the semantic
detection-by-segmentation network. SMCN learns the pancreas and mass
segmentation task and builds an anatomical correspondence-aware organ mesh
model by progressively deforming a pancreas prototype on the raw segmentation
mask (i.e., mask-to-mesh). A new graph-based residual convolutional network
(Graph-ResNet), whose nodes fuse the information of the mesh model and feature
vectors extracted from the segmentation network, is developed to produce the
patient-level differential classification results. Extensive experiments on 661
patients' CT scans (five phases per patient) show that SMCN can improve the
mass segmentation and detection accuracy compared to the strong baseline method
nnUNet (e.g., for nonPDAC, Dice: 0.611 vs. 0.478; detection rate: 89% vs. 70%),
achieve similar sensitivity and specificity in differentiating PDAC and nonPDAC
as expert radiologists (i.e., 94% and 90%), and obtain results comparable to a
multimodality test [20] that combines clinical, imaging, and molecular testing
for clinical management of patients.
