  One of the distinct characteristics in radiologists' reading of
multiparametric prostate MR scans, using reporting systems such as PI-RADS
v2.1, is to score individual types of MR modalities, T2-weighted,
diffusion-weighted, and dynamic contrast-enhanced, and then combine these
image-modality-specific scores using standardised decision rules to predict the
likelihood of clinically significant cancer. This work aims to demonstrate that
it is feasible for low-dimensional parametric models to model such decision
rules in the proposed Combiner networks, without compromising the accuracy of
predicting radiologic labels: First, it is shown that either a linear mixture
model or a nonlinear stacking model is sufficient to model PI-RADS decision
rules for localising prostate cancer. Second, parameters of these (generalised)
linear models are proposed as hyperparameters, to weigh multiple networks that
independently represent individual image modalities in the Combiner network
training, as opposed to end-to-end modality ensemble. A HyperCombiner network
is developed to train a single image segmentation network that can be
conditioned on these hyperparameters during inference, for much improved
efficiency. Experimental results based on data from 850 patients, for the
application of automating radiologist labelling multi-parametric MR, compare
the proposed combiner networks with other commonly-adopted end-to-end networks.
Using the added advantages of obtaining and interpreting the modality combining
rules, in terms of the linear weights or odds-ratios on individual image
modalities, three clinical applications are presented for prostate cancer
segmentation, including modality availability assessment, importance
quantification and rule discovery.
