  Understanding the severity of conditions shown in images in medical diagnosis
is crucial, serving as a key guide for clinical assessment, treatment, as well
as evaluating longitudinal progression. This paper proposes Con- PrO: a novel
representation learning method for severity assessment in medical images using
Contrastive learningintegrated Preference Optimization. Different from
conventional contrastive learning methods that maximize the distance between
classes, ConPrO injects into the latent vector the distance preference
knowledge between various severity classes and the normal class. We
systematically examine the key components of our framework to illuminate how
contrastive prediction tasks acquire valuable representations. We show that our
representation learning framework offers valuable severity ordering in the
feature space while outperforming previous state-of-the-art methods on
classification tasks. We achieve a 6% and 20% relative improvement compared to
a supervised and a self-supervised baseline, respectively. In addition, we
derived discussions on severity indicators and related applications of
preference comparison in the medical domain.
