<p>If the question is <em>'Will it start to rain based on this cloud cover?</em>' and the answer is a binary <em>'yes'</em> or <em>'no'</em> and the subsequent question is <em>'How sure are you about that?'</em>, then the answer to your question</p>

<blockquote>
  <p>Does the confidence rating reflect their estimate of likelihood of
  rain, or rather the likelihood that the choice is correct?</p>
</blockquote>

<p>is the latter: it reflects t<strong>he likelihood that the choice is correct</strong>, because when they say <em>'no it will not rain'</em>, with a confidence of 60%, then the chance it will rain is 40% and not 60%. Hence, it reflects the confidence they have in their answer.  </p>

CC BY-SA 4.0
2018-09-29T15:42:04.470Z
+1 This is true - so long as confidence ratings are limited to 50% or above.  Also see: https://en.wikipedia.org/wiki/Calibrated_probability_assessment: "... when a calibrated person says they are "80% confident" in each of 100 predictions they made, they will get about 80% of them correct."
Arnon Weinberg
CC BY-SA 4.0
2018-09-28T14:25:37.807Z
2018-09-28T14:25:37.807Z
AliceD
<p>Does a confidence rating included in a post task measure the likelihood of correctness to the answer or likelihood of occurrence of an event.</p>

<p>To give an example: participants are given a picture of clouds and they are provided with the binary question if they think it might start raining; the answer is thus either a '<em>yes</em>' or a '<em>no</em>'. Then they have to give an estimate of how confident they are of their decisions on a rating scale that indicates how confident they were; for example they might say '<em>70%</em>'.   </p>

<p><strong>Does the confidence rating reflect their estimate of likelihood of rain, or rather the likelihood that the choice is correct?</strong></p>

<methodology><task-analysis><confidence>
Confidence rating in cognitive experiments
CSL
# Confidence rating in cognitive experiments

(Asked by: username_0 on 2018-09-27)

Does a confidence rating included in a post task measure the likelihood of correctness to the answer or likelihood of occurrence of an event.

To give an example: participants are given a picture of clouds and they are provided with the binary question if they think it might start raining; the answer is thus either a '*yes*' or a '*no*'. Then they have to give an estimate of how confident they are of their decisions on a rating scale that indicates how confident they were; for example they might say '*70%*'.

**Does the confidence rating reflect their estimate of likelihood of rain, or rather the likelihood that the choice is correct?**

**Tags:** methodology, task-analysis, confidence


---

## 1 answers

### Answer by username_1 on 2018-09-28. Score: 1

If the question is *'Will it start to rain based on this cloud cover?*' and the answer is a binary *'yes'* or *'no'* and the subsequent question is *'How sure are you about that?'*, then the answer to your question

> Does the confidence rating reflect their estimate of likelihood of
> rain, or rather the likelihood that the choice is correct?

is the latter: it reflects t**he likelihood that the choice is correct**, because when they say *'no it will not rain'*, with a confidence of 60%, then the chance it will rain is 40% and not 60%. Hence, it reflects the confidence they have in their answer.

> **Comments:**
> **username_2**: +1 This is true - so long as confidence ratings are limited to 50% or above.  Also see: https://en.wikipedia.org/wiki/Calibrated_probability_assessment: "... when a calibrated person says they are "80% confident" in each of 100 predictions they made, they will get about 80% of them correct."  

---

