  Causal reasoning is the primary bottleneck that Large Language Models (LLMs)
must overcome to attain human-level intelligence. To address this, we introduce
the Causal Chain of Prompting (C2P) as the first reasoning framework that
equips current LLMs with causal reasoning capabilities. C2P operates
autonomously, avoiding reliance on external tools or modules during both the
causal learning and reasoning phases, and can be seamlessly implemented during
the training or fine-tuning of LLMs. Experimental results across various
benchmark datasets demonstrate a significant improvement in causal learning and
subsequent reasoning accuracy of LLMs. We illustrate how C2P enhances LLMs'
ability to causally reason in real-world scenarios, addressing complex problems
in fields such as healthcare, medicine, economics, education, social sciences,
environmental science, and marketing. With few-shot learning, GPT-4 Turbo using
C2P with as few as six examples achieves significant performance improvements,
boasting over a 33% increase in reasoning accuracy over the most
state-of-the-art LLMs, which perform nearly randomly in similar circumstances.
This demonstrates the transformative potential of integrating C2P into LLM
training or fine-tuning processes, thereby empowering these models with
advanced causal reasoning capabilities.
