  Background and Objective: Bladder cancer is a common malignant urinary
carcinoma, with muscle-invasive and non-muscle-invasive as its two major
subtypes. This paper aims to achieve automated bladder cancer invasiveness
localization and classification based on MRI. Method: Different from previous
efforts that segment bladder wall and tumor, we propose a novel end-to-end
multi-scale multi-task spatial feature encoder network (MM-SFENet) for locating
and classifying bladder cancer, according to the classification criteria of the
spatial relationship between the tumor and bladder wall. First, we built a
backbone with residual blocks to distinguish bladder wall and tumor; then, a
spatial feature encoder is designed to encode the multi-level features of the
backbone to learn the criteria. Results: We substitute Smooth-L1 Loss with IoU
Loss for multi-task learning, to improve the accuracy of the classification
task. By testing a total of 1287 MRIs collected from 98 patients at the
hospital, the mAP and IoU are used as the evaluation metrics. The experimental
result could reach 93.34\% and 83.16\% on test set. Conclusions: The
experimental result demonstrates the effectiveness of the proposed MM-SFENet on
the localization and classification of bladder cancer. It may provide an
effective supplementary diagnosis method for bladder cancer staging.
