  Numerous benchmarks aim to evaluate the capabilities of Large Language Models
(LLMs) for causal inference and reasoning. However, many of them can likely be
solved through the retrieval of domain knowledge, questioning whether they
achieve their purpose. In this review, we present a comprehensive overview of
LLM benchmarks for causality. We highlight how recent benchmarks move towards a
more thorough definition of causal reasoning by incorporating interventional or
counterfactual reasoning. We derive a set of criteria that a useful benchmark
or set of benchmarks should aim to satisfy. We hope this work will pave the way
towards a general framework for the assessment of causal understanding in LLMs
and the design of novel benchmarks.
