  Ordinal regression is a fundamental problem within the field of computer
vision, with customised well-trained models on specific tasks. While
pre-trained vision-language models (VLMs) have exhibited impressive performance
on various vision tasks, their potential for ordinal regression has received
less exploration. In this study, we first investigate CLIP's potential for
ordinal regression, from which we expect the model could generalise to
different ordinal regression tasks and scenarios. Unfortunately, vanilla CLIP
fails on this task, since current VLMs have a well-documented limitation of
encapsulating compositional concepts such as number sense. We propose a simple
yet effective method called NumCLIP to improve the quantitative understanding
of VLMs. We disassemble the exact image to number-specific text matching
problem into coarse classification and fine prediction stages. We discretize
and phrase each numerical bin with common language concept to better leverage
the available pre-trained alignment in CLIP. To consider the inherent
continuous property of ordinal regression, we propose a novel fine-grained
cross-modal ranking-based regularisation loss specifically designed to keep
both semantic and ordinal alignment in CLIP's feature space. Experimental
results on three general ordinal regression tasks demonstrate the effectiveness
of NumCLIP, with 10% and 3.83% accuracy improvement on historical image dating
and image aesthetics assessment task, respectively. Code is publicly available
at https://github.com/xmed-lab/NumCLIP.
