  The causal capabilities of large language models (LLMs) is a matter of
significant debate, with critical implications for the use of LLMs in
societally impactful domains such as medicine, science, law, and policy. We
further our understanding of LLMs and their causal implications, considering
the distinctions between different types of causal reasoning tasks, as well as
the entangled threats of construct and measurement validity. LLM-based methods
establish new state-of-the-art accuracies on multiple causal benchmarks.
Algorithms based on GPT-3.5 and 4 outperform existing algorithms on a pairwise
causal discovery task (97%, 13 points gain), counterfactual reasoning task
(92%, 20 points gain), and actual causality (86% accuracy in determining
necessary and sufficient causes in vignettes). At the same time, LLMs exhibit
unpredictable failure modes and we provide some techniques to interpret their
robustness.
  Crucially, LLMs perform these causal tasks while relying on sources of
knowledge and methods distinct from and complementary to non-LLM based
approaches. Specifically, LLMs bring capabilities so far understood to be
restricted to humans, such as using collected knowledge to generate causal
graphs or identifying background causal context from natural language. We
envision LLMs to be used alongside existing causal methods, as a proxy for
human domain knowledge and to reduce human effort in setting up a causal
analysis, one of the biggest impediments to the widespread adoption of causal
methods. We also see existing causal methods as promising tools for LLMs to
formalize, validate, and communicate their reasoning especially in high-stakes
scenarios.
  In capturing common sense and domain knowledge about causal mechanisms and
supporting translation between natural language and formal methods, LLMs open
new frontiers for advancing the research, practice, and adoption of causality.
