  This work aims toward an application of natural language processing (NLP)
technology for automatic verification of causal graphs using text sources. A
causal graph is often derived from unsupervised causal discovery methods and
requires manual evaluation from human experts. NLP technologies, i.e., Large
Language Models (LLMs) such as BERT and ChatGPT, can potentially be used to
verify the resulted causal graph by predicting if causal relation can be
observed between node pairs based on the textual context. In this work, we
compare the performance of two types of NLP models: (1) Pre-trained language
models fine-tuned for causal relation classification task and, (2) prompt-based
LLMs. Contrasted to previous studies where prompt-based LLMs work relatively
well over a set of diverse tasks, preliminary experiments on biomedical and
open-domain datasets suggest that the fine-tuned models far outperform the
prompt-based LLMs, up to 20.5 points improvement of F1 score. We shared the
code and the pre-processed datasets in our repository.
