  At the core of causal inference lies the challenge of determining reliable
causal graphs solely based on observational data. Since the well-known backdoor
criterion depends on the graph, any errors in the graph can propagate
downstream to effect inference. In this work, we initially show that complete
graph information is not necessary for causal effect inference; the topological
order over graph variables (causal order) alone suffices. Further, given a node
pair, causal order is easier to elicit from domain experts compared to graph
edges since determining the existence of an edge can depend extensively on
other variables. Interestingly, we find that the same principle holds for Large
Language Models (LLMs) such as GPT-3.5-turbo and GPT-4, motivating an automated
method to obtain causal order (and hence causal effect) with LLMs acting as
virtual domain experts. To this end, we employ different prompting strategies
and contextual cues to propose a robust technique of obtaining causal order
from LLMs. Acknowledging LLMs' limitations, we also study possible techniques
to integrate LLMs with established causal discovery algorithms, including
constraint-based and score-based methods, to enhance their performance.
Extensive experiments demonstrate that our approach significantly improves
causal ordering accuracy as compared to discovery algorithms, highlighting the
potential of LLMs to enhance causal inference across diverse fields.
