<p><a href="https://psychology.stackexchange.com/a/1302">@CHCH</a> has provided a good broad overview, but I thought I would also append some specific experiments that are considered to be a weakness of Bayesian models. The whole theme of this answer is an extension of Tversky and Kahneman's program of rationality-violation. All of these experiments <em>can</em> be fitted by <em>some</em> Bayesian-ish just-so model of the sort <a href="https://psycnet.apa.org/record/2012-10601-001" rel="nofollow noreferrer">Bowers &amp; Davis (2012)</a> discuss. These <strong>just-so models</strong> usually requires an unreasonably large state space (lots of free parameters), or grafting on unnatural heuristic-and-biases mechanisms for each individual experiment without any guidance from Bayesianism. I will present my narrative as an annotated bibliography.</p>
<blockquote>
<p>Shafir, E. &amp; Tversky, A. (1992) Thinking through uncertainty: nonconsequential reasoning and choice. <em>Cognitive Psychology</em> 24: 449-474.</p>
<p>Townsend, J.T., Silva, K.M., Spencer-Smith, J., &amp; Wenger, M. (2000) Exploring the relations between categorization and decision making with regard to realistic face stimuli. <em>Pragmatics and Cognition</em> 8: 83-105.</p>
<p>Tversky, A., &amp; Shafir, E. (1992) The disjunction effect in choice under uncertainty. <em>Psychological Science</em> 3: 305-309.</p>
</blockquote>
<p>As mentioned in the question, and <a href="https://cogsci.stackexchange.com/q/251/29">elsewhere on the site</a>, <strong>violation of the sure-thing principle</strong> is one of the most popular examples of departures from Bayesianism. Given a random variable that can have only two possible outcomes A and B, probability requires <span class="math-container">$p(X)$</span> to be between <span class="math-container">$p(X|A)$</span> and <span class="math-container">$p(X|B)$</span>. A violation is when <span class="math-container">$p(X) &gt; p(X|A)$</span> <em>and</em> <span class="math-container">$p(X) &gt; p(X|B)$</span> (or both <span class="math-container">$&lt;$</span> instead). ST showed the violation in a game-theoretic setting, TS in a two-stage gambling task, and TSS-SW in a face-categorization task.</p>
<blockquote>
<p>Tversky, A., &amp; Kahneman, D. (1983). Extensional versus intuitive reasoning: the conjuctive fallacy in probability judgement. <em>Psychology Review</em> 101: 547-567.</p>
</blockquote>
<p>In this study, TK described Linda and then asked the participant to make a probability judgement <span class="math-container">$p(\text{BT})$</span> of Linda being a bank-teller, or a probability judgement <span class="math-container">$p(\text{BT} \;\&amp;\; \text{F})$</span> of Linda being a bank-teller <em>and</em> a feminist. In any Bayesian model (without some grafted mechanisms or weird latent variables) you need to have <span class="math-container">$p(\text{BT}) \geq p(\text{BT} \;\&amp;\; \text{F})$</span>, but the participants judged <span class="math-container">$p(\text{BT}) &lt; p(\text{BT} \;\&amp;\; \text{F})$</span> and committed a <strong>conjunction fallacy</strong>.</p>
<blockquote>
<p>Gavanski, I., &amp; Roskos-Ewoldsen, D.R. (1991) <a href="https://psycnet.apa.org/record/1991-33124-001" rel="nofollow noreferrer">Representativeness and conjoint probability</a>. <em>Journal of Personality and Social Psychology</em> 61(2): 181-194.</p>
</blockquote>
<p>TK's ad-hoc explanation for this is the <em>representative heuristic</em>. GR-E recreate the fallacy in a setting where they show that the <strong>ad-hoc heuristic is not sufficient</strong>. This points out the weakness of ad-hoc fixes to Bayesian-ish approaches.</p>
<blockquote>
<p>Sides, A., Osherson, D., Bonini, N., and Viale, R. (2002) <a href="https://doi.org/10.3758/BF03195280" rel="nofollow noreferrer">On the reality of the conjunction fallacy</a>. <em>Memory &amp; Cognition</em> 30(2): 191-198.</p>
</blockquote>
<p>It is natural to suspect that TK's result could be an artifact of participants not understanding the modern concept of probability. SOBV account for this by using a betting paradigm that uses probabilities implicitly instead of asking participants to report numeric values. SOBV show that the conjunction fallacy is <strong>independent of numeric probability reporting</strong> and thus an intrinsic 'error'.</p>
<blockquote>
<p>Feldman, J.M. &amp; Lynch, J.G. (1988) Self-generated validity and other effects of measurement on belief, attitude, intention, and behavior. <em>Journal of Applied Psychology</em> 73: 421-435.</p>
<p>Moore, D.W. (2002) Measuring new types of question-order effects. <em>Public Opinion Quarterly</em> 66: 80-91</p>
<p>Schuman, H., &amp; Presser, S. (1981) Questions and answers in attitude surveys: experiments on question form, wording, and content</p>
</blockquote>
<p>When conducting a questionnaire, the order questions are asked in changes the resulting probability judgements. For a purely Bayesian approach, the mutual probability of asking A then B and getting a specific outcome is <span class="math-container">$p(A)p(B|A) = p(A \&amp; B) = p(B \&amp; A) = p(B)p(A|B)$</span>. This would suggest that the order questions are asked shouldn't matter, so we have a <strong>failure of commutativity</strong>.</p>
<blockquote>
<p>Hogarth, R.M. &amp; Einhorn, H.J. (1992) Order effects in belief updating: the belief-adjustment model. <em>Cognitive Psycholog</em> 24: 1-55.</p>
<p>Shanteau, J.C. (1970) An additivity model for sequential decision making. <em>Journal of Experimental Psychology</em>. 85: 181-191.</p>
</blockquote>
<p>The <strong>order effect</strong> is not confined to questions, but also to integrating evidence. The strongest point of Bayesianism is a clear theory of how to update hypotheses, given evidence. Unfortunately, for Bayes' rule <span class="math-container">$p(H|A \&amp; B) = p(H|B \&amp; A)$</span>, but HE &amp; S show that for humans this is not always the case and present heuristic-and-biases alternatives.</p>
<blockquote>
<p>Berges, G.R., Chapman, G.B., Levy, B.T., Ely, J.W., &amp; Oppliger, R.A. (1998) Clinical diagnosis and order information. <em>Medical Decision Making</em> 18: 412-417.</p>
<p>McKenzie, C.R.M., Lee, S.M., &amp; Chen, K.K. (2002) When negative evidence increases confidence: change in belief after hearing two sides of a dispute. <em>Journal of Behavioral Decision Making</em> 15: 1-18.</p>
</blockquote>
<p>It might be tempting to suspect that these order effects are confined to artificial laboratory settings. Unfortunately, they <strong>appear in natural settings</strong> associated with rational decision making such as BCLEO's study of medical diagnosis and MLC's of dispute mediation in a jury.</p>
<blockquote>
<p>Aerts, D. &amp; Sozzo, S. (2011) <a href="https://doi.org/10.1007/978-3-642-24971-6_12" rel="nofollow noreferrer">Quantum Structure in Cognition: Why and How Concepts Are Entangled</a>. <em>Quantum Interaction</em> 7052: 116-127.</p>
</blockquote>
<p>AS studied membership judgements for pairs of <strong>concept combinations</strong>, and found among their participants found forms of dependence between concept pairs that violated <a href="https://en.wikipedia.org/wiki/Bell%27s_theorem" rel="nofollow noreferrer">Bell's theorem</a> and thus could not be fit by any reasonable classical joint distribution over the concept combinations.</p>

CC BY-SA 4.0
2012-09-12T16:21:28.630Z
2022-05-29T06:23:06.647Z
2022-05-29T06:23:06.647Z
Artem Kaznatcheev
<p>There is currently a lot of debate surrounding what questions Bayesian modeling is appropriate for answering within cognitive science, as well what makes a "poor model."  Unfortunately these become extremely thorny issues very quickly, partly because what is called "bayesian modeling" actually refers to a rather heterogeneous set of approaches and philosophies, and partly because different models serve very different purposes, making the definition of a "poor model" very difficult to agree on.</p>

<p>Nonetheless, I can strongly recommend two papers that deal with these issues in depth.  The first is Jones &amp; Love's 2011 Behavioral &amp; Brain Sciences paper "Bayesian Fundamentalism or Enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition."  Fortunately, BBS articles include open peer commentary - so you can see the reaction of the larger field to their claims.  </p>

<p>The second paper is Bowers &amp; Davis's 2012 Psychonomic Bulletin &amp; Review article titled "Bayesian just-so stories in psychology and neuroscience." These papers both make very credible arguments that Bayesian models are actually "poor models" even when they would claim to be good; both papers highlight a number of phenomena that are difficult to fit with Bayesian models without resorting to the same kinds of parameter twiddling that Bayesians tend to criticize other modelers for. </p>

<p>Finally, there is a paper that just came out today describing the failure of a Bayesian model to capture some phenomena in the domain of causal reasoning.  This has been traditionally understood as a relative strong suit of Bayesian modeling, making the findings of this paper somewhat striking.  I have not read it, but it's in press by Bes, Sloman, Lucas &amp; Raufaste at Cognitive Science, titled "Non-Bayesian Inference: Causal Structure Trumps Correlation."</p>

<h3>References</h3>

<ul>
<li>Bowers, J.S. &amp; Davis, C.J. (2012). Bayesian just-so stories in psychology and neuroscience.. Psychological bulletin, 138, 389.<a href="http://www.yangzhiping.com/files/pubs/389.pdf">PDF</a></li>
<li>Jones, M. &amp; Love, B.C. (2011). Bayesian Fundamentalism or Enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behavioral and Brain Sciences, 34, 169-188. <a href="http://www.psych.unimelb.edu.au/people/staff/allpubs/SewellD/Jones2011_BBS.pdf">PDF</a></li>
<li>Bes, B., Sloman, S., Lucas, C.G. &amp; Raufaste, E. (2012). Non-Bayesian inference: Causal structure trumps correlation. Cognitive Science. <a href="https://1b7a2906-a-62cb3a1a-s-sites.googlegroups.com/site/slomanlab/BesSlomanLucasRaufaste.pdf?attachauth=ANoY7cqTRlah9D6Qf7XORUUtlzhfDRkZMS7n0exl6h5qlFUCErP8Q8MHjix8U9Hi4_difgmDQeFQ8vtVxl7l3f1KqpFd8be8J3augvkNl5dLt5t7uRmsggk-KUJJa3xJDLGXHkMvDBXFrsCeCzFuptvlY32vi71hsfjCO8zNPKYvMAjN8rWUyxpJXyeq7zV8Tce_Tmu4fzIlvNabQhwIXnvKUmHfhuYUgXQ73ZgonZbibAlhRw0AHIU%3D&amp;attredirects=0">PDF</a></li>
</ul>

CC BY-SA 3.0
2012-06-26T23:16:24.673Z
Welcome to CogSci. Awesome answer, thanks! I was actually going to self-answer with some ideas from another BBS issue. Is it possible to expand this answer a little bit by mentioning some specific weaknesses the 2011 BBS comments stressed? Also, you should consider contributing an answer to [this question](http://cogsci.stackexchange.com/q/599/29), great to have you on the site.
Artem Kaznatcheev
CC BY-SA 3.0
2012-06-27T00:17:23.443Z
@CHCH: Looks like you are going to be a very helpful member and good source of information.
Greg McNulty
CC BY-SA 3.0
2012-06-27T02:44:05.223Z
I agree with the others!  Welcome aboard!
Chuck Sherrington
CC BY-SA 3.0
2012-06-26T22:37:28.153Z
2012-06-27T01:05:52.133Z
2012-06-27T01:05:52.133Z
CHCH
<p><a href="http://lesswrong.com/lw/1to/what_is_bayesianism/" rel="nofollow noreferrer">Bayesianism</a> has been a relatively successful paradigm for modeling decision-making. However, not every psychologist is a bayesian, and there are tasks such as the Tversky &amp; Shafir (1992) experiments that are hard to <a href="https://cogsci.stackexchange.com/q/251/29">reconcile with the bayesian approach</a>. </p>

<p><strong>Is there a list (or suvery-paper) of decision-making experiments that are hard to account for with the probabilistic approach?</strong> What are these experiments? What are some of their common features?</p>

<hr>

<h3>References and related questions</h3>

<ul>
<li><p>Shafir, E., &amp; Tversky, A. (1992). Thinking through uncertainty: Nonconsequential reasoning and choice. Cognitive Psychology, 24, 449-474. PDF</p></li>
<li><p>Tversky, A., &amp; Shafir, E. (1992). The disjunction effect in choice under uncertainty. Psychological Science, 3, 305-309.</p></li>
<li><p><a href="https://cogsci.stackexchange.com/q/599/29">What are some of the drawbacks to probabilistic models of cognition?</a></p></li>
<li><p><a href="https://cogsci.stackexchange.com/q/720/29">Introductory resources on bayesian modeling for cognitive sciences</a></p></li>
<li><p><a href="https://cogsci.stackexchange.com/q/1278/29">How can the success of Bayesian models be reconciled with demonstrations of heuristic and biased reasoning?</a></p></li>
</ul>

<reference-request><decision-making><experimental-psychology><rationality><bayesian>
What tasks does Bayesian decision-making model poorly?
Artem Kaznatcheev
# What tasks does Bayesian decision-making model poorly?

(Asked by: username_0 on 2012-06-26)

[Bayesianism](http://lesswrong.com/lw/1to/what_is_bayesianism/) has been a relatively successful paradigm for modeling decision-making. However, not every psychologist is a bayesian, and there are tasks such as the Tversky & Shafir (1992) experiments that are hard to [reconcile with the bayesian approach](https://cogsci.stackexchange.com/q/251/29).

**Is there a list (or suvery-paper) of decision-making experiments that are hard to account for with the probabilistic approach?** What are these experiments? What are some of their common features?

---

### References and related questions

* Shafir, E., & Tversky, A. (1992). Thinking through uncertainty: Nonconsequential reasoning and choice. Cognitive Psychology, 24, 449-474. PDF
* Tversky, A., & Shafir, E. (1992). The disjunction effect in choice under uncertainty. Psychological Science, 3, 305-309.
* [What are some of the drawbacks to probabilistic models of cognition?](https://cogsci.stackexchange.com/q/599/29)
* [Introductory resources on bayesian modeling for cognitive sciences](https://cogsci.stackexchange.com/q/720/29)
* [How can the success of Bayesian models be reconciled with demonstrations of heuristic and biased reasoning?](https://cogsci.stackexchange.com/q/1278/29)

**Tags:** reference-request, decision-making, experimental-psychology, rationality, bayesian


---

## 2 answers

### Answer by username_1 on 2012-06-26. Score: 16 | Accepted answer

There is currently a lot of debate surrounding what questions Bayesian modeling is appropriate for answering within cognitive science, as well what makes a "poor model." Unfortunately these become extremely thorny issues very quickly, partly because what is called "bayesian modeling" actually refers to a rather heterogeneous set of approaches and philosophies, and partly because different models serve very different purposes, making the definition of a "poor model" very difficult to agree on.

Nonetheless, I can strongly recommend two papers that deal with these issues in depth. The first is Jones & Love's 2011 Behavioral & Brain Sciences paper "Bayesian Fundamentalism or Enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition." Fortunately, BBS articles include open peer commentary - so you can see the reaction of the larger field to their claims.

The second paper is Bowers & Davis's 2012 Psychonomic Bulletin & Review article titled "Bayesian just-so stories in psychology and neuroscience." These papers both make very credible arguments that Bayesian models are actually "poor models" even when they would claim to be good; both papers highlight a number of phenomena that are difficult to fit with Bayesian models without resorting to the same kinds of parameter twiddling that Bayesians tend to criticize other modelers for.

Finally, there is a paper that just came out today describing the failure of a Bayesian model to capture some phenomena in the domain of causal reasoning. This has been traditionally understood as a relative strong suit of Bayesian modeling, making the findings of this paper somewhat striking. I have not read it, but it's in press by Bes, Sloman, Lucas & Raufaste at Cognitive Science, titled "Non-Bayesian Inference: Causal Structure Trumps Correlation."

### References

* Bowers, J.S. & Davis, C.J. (2012). Bayesian just-so stories in psychology and neuroscience.. Psychological bulletin, 138, 389.[PDF](http://www.yangzhiping.com/files/pubs/389.pdf)
* Jones, M. & Love, B.C. (2011). Bayesian Fundamentalism or Enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behavioral and Brain Sciences, 34, 169-188. [PDF](http://www.psych.unimelb.edu.au/people/staff/allpubs/SewellD/Jones2011_BBS.pdf)
* Bes, B., Sloman, S., Lucas, C.G. & Raufaste, E. (2012). Non-Bayesian inference: Causal structure trumps correlation. Cognitive Science. [PDF](https://1b7a2906-a-62cb3a1a-s-sites.googlegroups.com/site/slomanlab/BesSlomanLucasRaufaste.pdf?attachauth=ANoY7cqTRlah9D6Qf7XORUUtlzhfDRkZMS7n0exl6h5qlFUCErP8Q8MHjix8U9Hi4_difgmDQeFQ8vtVxl7l3f1KqpFd8be8J3augvkNl5dLt5t7uRmsggk-KUJJa3xJDLGXHkMvDBXFrsCeCzFuptvlY32vi71hsfjCO8zNPKYvMAjN8rWUyxpJXyeq7zV8Tce_Tmu4fzIlvNabQhwIXnvKUmHfhuYUgXQ73ZgonZbibAlhRw0AHIU%3D&attredirects=0)

> **Comments:**
> **username_0**: Welcome to CogSci. Awesome answer, thanks! I was actually going to self-answer with some ideas from another BBS issue. Is it possible to expand this answer a little bit by mentioning some specific weaknesses the 2011 BBS comments stressed? Also, you should consider contributing an answer to [this question](http://cogsci.stackexchange.com/q/599/29), great to have you on the site.  
> **username_2**: @username_1: Looks like you are going to be a very helpful member and good source of information.  
> **username_3**: I agree with the others!  Welcome aboard!  

---

### Answer by username_0 on 2012-09-12. Score: 10

[@username_1](https://psychology.stackexchange.com/a/1302) has provided a good broad overview, but I thought I would also append some specific experiments that are considered to be a weakness of Bayesian models. The whole theme of this answer is an extension of Tversky and Kahneman's program of rationality-violation. All of these experiments *can* be fitted by *some* Bayesian-ish just-so model of the sort [Bowers & Davis (2012)](https://psycnet.apa.org/record/2012-10601-001) discuss. These **just-so models** usually requires an unreasonably large state space (lots of free parameters), or grafting on unnatural heuristic-and-biases mechanisms for each individual experiment without any guidance from Bayesianism. I will present my narrative as an annotated bibliography.

> Shafir, E. & Tversky, A. (1992) Thinking through uncertainty: nonconsequential reasoning and choice. *Cognitive Psychology* 24: 449-474.
>
> Townsend, J.T., Silva, K.M., Spencer-Smith, J., & Wenger, M. (2000) Exploring the relations between categorization and decision making with regard to realistic face stimuli. *Pragmatics and Cognition* 8: 83-105.
>
> Tversky, A., & Shafir, E. (1992) The disjunction effect in choice under uncertainty. *Psychological Science* 3: 305-309.

As mentioned in the question, and [elsewhere on the site](https://cogsci.stackexchange.com/q/251/29), **violation of the sure-thing principle** is one of the most popular examples of departures from Bayesianism. Given a random variable that can have only two possible outcomes A and B, probability requires $p(X)$ to be between $p(X|A)$ and $p(X|B)$. A violation is when $p(X) > p(X|A)$ *and* $p(X) > p(X|B)$ (or both $<$ instead). ST showed the violation in a game-theoretic setting, TS in a two-stage gambling task, and TSS-SW in a face-categorization task.

> Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: the conjuctive fallacy in probability judgement. *Psychology Review* 101: 547-567.

In this study, TK described Linda and then asked the participant to make a probability judgement $p(\text{BT})$ of Linda being a bank-teller, or a probability judgement $p(\text{BT} \;\&\; \text{F})$ of Linda being a bank-teller *and* a feminist. In any Bayesian model (without some grafted mechanisms or weird latent variables) you need to have $p(\text{BT}) \geq p(\text{BT} \;\&\; \text{F})$, but the participants judged $p(\text{BT}) < p(\text{BT} \;\&\; \text{F})$ and committed a **conjunction fallacy**.

> Gavanski, I., & Roskos-Ewoldsen, D.R. (1991) [Representativeness and conjoint probability](https://psycnet.apa.org/record/1991-33124-001). *Journal of Personality and Social Psychology* 61(2): 181-194.

TK's ad-hoc explanation for this is the *representative heuristic*. GR-E recreate the fallacy in a setting where they show that the **ad-hoc heuristic is not sufficient**. This points out the weakness of ad-hoc fixes to Bayesian-ish approaches.

> Sides, A., Osherson, D., Bonini, N., and Viale, R. (2002) [On the reality of the conjunction fallacy](https://doi.org/10.3758/BF03195280). *Memory & Cognition* 30(2): 191-198.

It is natural to suspect that TK's result could be an artifact of participants not understanding the modern concept of probability. SOBV account for this by using a betting paradigm that uses probabilities implicitly instead of asking participants to report numeric values. SOBV show that the conjunction fallacy is **independent of numeric probability reporting** and thus an intrinsic 'error'.

> Feldman, J.M. & Lynch, J.G. (1988) Self-generated validity and other effects of measurement on belief, attitude, intention, and behavior. *Journal of Applied Psychology* 73: 421-435.
>
> Moore, D.W. (2002) Measuring new types of question-order effects. *Public Opinion Quarterly* 66: 80-91
>
> Schuman, H., & Presser, S. (1981) Questions and answers in attitude surveys: experiments on question form, wording, and content

When conducting a questionnaire, the order questions are asked in changes the resulting probability judgements. For a purely Bayesian approach, the mutual probability of asking A then B and getting a specific outcome is $p(A)p(B|A) = p(A \& B) = p(B \& A) = p(B)p(A|B)$. This would suggest that the order questions are asked shouldn't matter, so we have a **failure of commutativity**.

> Hogarth, R.M. & Einhorn, H.J. (1992) Order effects in belief updating: the belief-adjustment model. *Cognitive Psycholog* 24: 1-55.
>
> Shanteau, J.C. (1970) An additivity model for sequential decision making. *Journal of Experimental Psychology*. 85: 181-191.

The **order effect** is not confined to questions, but also to integrating evidence. The strongest point of Bayesianism is a clear theory of how to update hypotheses, given evidence. Unfortunately, for Bayes' rule $p(H|A \& B) = p(H|B \& A)$, but HE & S show that for humans this is not always the case and present heuristic-and-biases alternatives.

> Berges, G.R., Chapman, G.B., Levy, B.T., Ely, J.W., & Oppliger, R.A. (1998) Clinical diagnosis and order information. *Medical Decision Making* 18: 412-417.
>
> McKenzie, C.R.M., Lee, S.M., & Chen, K.K. (2002) When negative evidence increases confidence: change in belief after hearing two sides of a dispute. *Journal of Behavioral Decision Making* 15: 1-18.

It might be tempting to suspect that these order effects are confined to artificial laboratory settings. Unfortunately, they **appear in natural settings** associated with rational decision making such as BCLEO's study of medical diagnosis and MLC's of dispute mediation in a jury.

> Aerts, D. & Sozzo, S. (2011) [Quantum Structure in Cognition: Why and How Concepts Are Entangled](https://doi.org/10.1007/978-3-642-24971-6_12). *Quantum Interaction* 7052: 116-127.

AS studied membership judgements for pairs of **concept combinations**, and found among their participants found forms of dependence between concept pairs that violated [Bell's theorem](https://en.wikipedia.org/wiki/Bell%27s_theorem) and thus could not be fit by any reasonable classical joint distribution over the concept combinations.


---

