  Large language models (LLMs) are increasingly applied to clinical
decision-making. However, their potential to exhibit bias poses significant
risks to clinical equity. Currently, there is a lack of benchmarks that
systematically evaluate such clinical bias in LLMs. While in downstream tasks,
some biases of LLMs can be avoided such as by instructing the model to answer
"I'm not sure...", the internal bias hidden within the model still lacks deep
studies. We introduce CLIMB (shorthand for A Benchmark of Clinical Bias in
Large Language Models), a pioneering comprehensive benchmark to evaluate both
intrinsic (within LLMs) and extrinsic (on downstream tasks) bias in LLMs for
clinical decision tasks. Notably, for intrinsic bias, we introduce a novel
metric, AssocMAD, to assess the disparities of LLMs across multiple demographic
groups. Additionally, we leverage counterfactual intervention to evaluate
extrinsic bias in a task of clinical diagnosis prediction. Our experiments
across popular and medically adapted LLMs, particularly from the Mistral and
LLaMA families, unveil prevalent behaviors with both intrinsic and extrinsic
bias. This work underscores the critical need to mitigate clinical bias and
sets a new standard for future evaluations of LLMs' clinical bias.
