  Regression plays an essential role in many medical imaging applications for
estimating various clinical risk or measurement scores. While training
strategies and loss functions have been studied for the deep neural networks in
medical image classification tasks, options for regression tasks are very
limited. One of the key challenges is that the high-dimensional feature
representation learned by existing popular loss functions like Mean Squared
Error or L1 loss is hard to interpret. In this paper, we propose a novel
Regression Metric Loss (RM-Loss), which endows the representation space with
the semantic meaning of the label space by finding a representation manifold
that is isometric to the label space. Experiments on two regression tasks, i.e.
coronary artery calcium score estimation and bone age assessment, show that
RM-Loss is superior to the existing popular regression losses on both
performance and interpretability. Code is available at
https://github.com/DIAL-RPI/Regression-Metric-Loss.
