  Osteoarthritis (OA) poses a global health challenge, demanding precise
diagnostic methods. Current radiographic assessments are time consuming and
prone to variability, prompting the need for automated solutions. The existing
deep learning models for OA assessment are unimodal single task systems and
they don't incorporate relevant text information such as patient demographics,
disease history, or physician reports. This study investigates employing Vision
Language Processing (VLP) models to predict OA severity using Xray images and
corresponding reports. Our method leverages Xray images of the knee and diverse
report templates generated from tabular OA scoring values to train a CLIP
(Contrastive Language Image PreTraining) style VLP model. Furthermore, we
incorporate additional contrasting captions to enforce the model to
discriminate between positive and negative reports. Results demonstrate the
efficacy of these models in learning text image representations and their
contextual relationships, showcase potential advancement in OA assessment, and
establish a foundation for specialized vision language models in medical
contexts.
