  This paper explores the causal reasoning of large language models (LLMs) to
enhance their interpretability and reliability in advancing artificial
intelligence. Despite the proficiency of LLMs in a range of tasks, their
potential for understanding causality requires further exploration. We propose
a novel causal attribution model that utilizes ``do-operators" for constructing
counterfactual scenarios, allowing us to systematically quantify the influence
of input numerical data and LLMs' pre-existing knowledge on their causal
reasoning processes. Our newly developed experimental setup assesses LLMs'
reliance on contextual information and inherent knowledge across various
domains. Our evaluation reveals that LLMs' causal reasoning ability mainly
depends on the context and domain-specific knowledge provided. In the absence
of such knowledge, LLMs can still maintain a degree of causal reasoning using
the available numerical data, albeit with limitations in the calculations. This
motivates the proposed fine-tuned LLM for pairwise causal discovery,
effectively leveraging both knowledge and numerical information.
