  Rectal cancer is one of the most common diseases and a major cause of
mortality. For deciding rectal cancer treatment plans, T-staging is important.
However, evaluating the index from preoperative MRI images requires high
radiologists' skill and experience. Therefore, the aim of this study is to
segment the mesorectum, rectum, and rectal cancer region so that the system can
predict T-stage from segmentation results. Generally, shortage of large and
diverse dataset and high quality annotation are known to be the bottlenecks in
computer aided diagnostics development. Regarding rectal cancer, advanced
cancer images are very rare, and per-pixel annotation requires high
radiologists' skill and time. Therefore, it is not feasible to collect
comprehensive disease patterns in a training dataset. To tackle this, we
propose two kinds of approaches of image synthesis-based late stage cancer
augmentation and semi-supervised learning which is designed for T-stage
prediction. In the image synthesis data augmentation approach, we generated
advanced cancer images from labels. The real cancer labels were deformed to
resemble advanced cancer labels by artificial cancer progress simulation. Next,
we introduce a T-staging loss which enables us to train segmentation models
from per-image T-stage labels. The loss works to keep inclusion/invasion
relationships between rectum and cancer region consistent to the ground truth
T-stage. The verification tests show that the proposed method obtains the best
sensitivity (0.76) and specificity (0.80) in distinguishing between over T3
stage and underT2. In the ablation studies, our semi-supervised learning
approach with the T-staging loss improved specificity by 0.13. Adding the image
synthesis-based data augmentation improved the DICE score of invasion cancer
area by 0.08 from baseline.
