  This study explores the potential of natural language models, including large
language models, to extract causal relations from medical texts, specifically
from Clinical Practice Guidelines (CPGs). The outcomes causality extraction
from Clinical Practice Guidelines for gestational diabetes are presented,
marking a first in the field. We report on a set of experiments using variants
of BERT (BioBERT, DistilBERT, and BERT) and using Large Language Models (LLMs),
namely GPT-4 and LLAMA2. Our experiments show that BioBERT performed better
than other models, including the Large Language Models, with an average
F1-score of 0.72. GPT-4 and LLAMA2 results show similar performance but less
consistency. We also release the code and an annotated a corpus of causal
statements within the Clinical Practice Guidelines for gestational diabetes.
