  Causality is essential for understanding complex systems, such as the
economy, the brain, and the climate. Constructing causal graphs often relies on
either data-driven or expert-driven approaches, both fraught with challenges.
The former methods, like the celebrated PC algorithm, face issues with data
requirements and assumptions of causal sufficiency, while the latter demand
substantial time and domain knowledge. This work explores the capabilities of
Large Language Models (LLMs) as an alternative to domain experts for causal
graph generation. We frame conditional independence queries as prompts to LLMs
and employ the PC algorithm with the answers. The performance of the LLM-based
conditional independence oracle on systems with known causal graphs shows a
high degree of variability. We improve the performance through a proposed
statistical-inspired voting schema that allows some control over false-positive
and false-negative rates. Inspecting the chain-of-thought argumentation, we
find causal reasoning to justify its answer to a probabilistic query. We show
evidence that knowledge-based CIT could eventually become a complementary tool
for data-driven causal discovery.
