  Calibration, which establishes the correlation between accuracy and model
confidence, is important for LLM development. We design three off-the-shelf
calibration methods based on self-consistency (Wang et al., 2022) for math
reasoning tasks. Evaluation on two popular benchmarks (GSM8K and MathQA) using
strong open-source LLMs (Mistral and LLaMA2), our methods better bridge model
confidence and accuracy than existing methods based on p(True) (Kadavath et
al., 2022) or logit (Kadavath et al., 2022).
