  This work presents comprehensive results to detect in the early stage the
pancreatic neuroendocrine tumors (PNETs), a group of endocrine tumors arising
in the pancreas, which are the second common type of pancreatic cancer, by
checking the abdominal CT scans. To the best of our knowledge, this task has
not been studied before as a computational task. To provide radiologists with
tumor locations, we adopt a segmentation framework to classify CT volumes by
checking if at least a sufficient number of voxels is segmented as tumors. To
quantitatively analyze our method, we collect and voxelwisely label a new
abdominal CT dataset containing $376$ cases with both arterial and venous
phases available for each case, in which $228$ cases were diagnosed with PNETs
while the remaining $148$ cases are normal, which is currently the largest
dataset for PNETs to the best of our knowledge. In order to incorporate rich
knowledge of radiologists to our framework, we annotate dilated pancreatic duct
as well, which is regarded as the sign of high risk for pancreatic cancer.
Quantitatively, our approach outperforms state-of-the-art segmentation networks
and achieves a sensitivity of $89.47\%$ at a specificity of $81.08\%$, which
indicates a potential direction to achieve a clinical impact related to cancer
diagnosis by earlier tumor detection.
