<p>Artem gave a very good answer, but I want to add one more weaknesses of probabilistic/Bayesian models: they are not mechanistic. This is related to Artem's point about neural grounding, but is a little different. The issue is that probabilistic models don't really provide insight into the underlying mechanism that produces the observed behavior -- if you ask the question "why does it work?", the Bayesian model's answer is "because cognition is Bayesian" and that's it. Models that allow structure (and behavior) to emerge from interactions of lower-level elements provide more insight into cognitive mechanisms (though not necessarily neural mechanisms). I think (at least some) connectionist and dynamic systems models try to do this. To read more about this, check out:</p>

<p>McClelland, J. L., Botvinick, M. M., Noelle, D. C., Plaut, D. C., Rogers, T.T., Seidenberg, M. S., and Smith, L. B. (2010). <a href="http://www-psych.stanford.edu/~jlm/papers/McCEtAl10TiCS_LettingStructureEmerge.pdf">Letting Structure Emerge: Connectionist and Dynamical Systems Approaches to Understanding Cognition</a>. <em>Trends in Cognitive Sciences</em>, 14, 348-356.</p>

CC BY-SA 3.0
2012-03-15T20:14:50.697Z
Griffiths et al in the same issue is a good counterpart to this article: http://www.psychology.adelaide.edu.au/personalpages/staff/amyperfors/papers/griffithsetal10tics-probabilisticmodelsofcognition.pdf
Jeff
CC BY-SA 3.0
2012-03-17T14:31:02.310Z
This is a good point! Although I suspect a bayesian would respond that it is not clear how connectionists or dynamic explanations are more mechanistic, since they tend to hide things in poorly understood 'emergence'.
Artem Kaznatcheev
CC BY-SA 3.0
2012-04-29T03:50:05.243Z
I have had similar sentiments, one of the reasons why I asked my question.
Vielle
CC BY-SA 3.0
2012-03-15T18:10:19.163Z
2012-03-15T18:10:19.163Z
Dan M.
<p>This should perhaps be a comment, but I don't have the reputation. The other two answers mention that a major drawback to the Bayesian approach is its lack of biological plausibility. However, see for instance:</p>

<blockquote>
  <p><a href="http://www.nature.com/neuro/journal/v9/n11/abs/nn1790.html">Bayesian inference with probabilistic population codes</a></p>
  
  <p>Ma, W.J. and Beck, J.M. and Latham, P.E. and Pouget, A.</p>
  
  <p><em>Nature Neuroscience</em>, 2006</p>
</blockquote>

<p>The authors propose a method by which populations of neurons may actually represent probability distributions. I don't know how convincing I find the construction, but it may be worth looking into if you're curious about the subject. </p>

<p>It may also be worth mentioning that while neuroscientists do seem to use the word "Bayesian" to refer generically to probabilistic approaches to reasoning, "Bayesian statistics" or "<a href="http://en.wikipedia.org/wiki/Bayesian_probability#Objective_and_subjective_Bayesian_probabilities">Bayesian probability</a>" is a bit more than Bayes' rule. By itself, Bayes' rule is <a href="http://en.wikipedia.org/wiki/Bayes%27_theorem#Statement_and_interpretation">just a mathematical identity in probability theory</a>. </p>

CC BY-SA 3.0
2012-04-03T19:14:17.913Z
I agree that this is more a comment than an answer, but it could be closer to an answer on [this question](http://cogsci.stackexchange.com/q/644/29). Also, your last paragraph is incredibly true, and I wish more researchers were conscious of this :D.
Artem Kaznatcheev
CC BY-SA 3.0
2012-04-03T03:01:19.833Z
2012-04-03T18:53:31.853Z
2012-04-03T18:53:31.853Z
yep
<p>Probabilistic approaches of this sort are usually referred to more specifically as the <strong>bayesian</strong> approach and Chater and Tanenbaum are definitely bayesians (I have not read much by Yuille and can't comment). <a href="http://lesswrong.com/lw/1to/what_is_bayesianism/" rel="nofollow noreferrer">Bayesianism</a> is more than just increasing in popularity and being encouraged; it is considered one of the big-4 approaches to cognitive-modeling, with the other 3 being: connectionism, rule-based, and dynamic systems. The bayesian approach has many positives and produced many great results, but since your question is about the drawbacks I will focus exclusively on that. Two major drawbacks are: neural grounding and rationality.</p>

<p><strong>Neural grounding</strong> is a weakness that plagues all of the big-4 and cognitive science in general. The idea is that as we build models of the mind, we want to eventually <a href="https://cogsci.stackexchange.com/q/90/29">ground them in the brain</a>; this is a standard feature of reductionism. The bayesian approach is often summarized as "probabilities over rules", and suffers from the same difficulty of neural grounding as the rule-based approach did. It is often not clear how the brain implements this sophisticated bayesian inference (but the field is well aware of this problem, and works hard to resolve it). Is this a game killer? Not really, connectionism is often considered the more 'biologically-plausible' alternative, but most popular connectionist models can be just as <a href="https://cogsci.stackexchange.com/q/214/29">easily questioned on their biological viability</a>. The issue can also be sidestepped completely by saying that we do wish to address cognition at a different level that biological implementation (sort of how thermodynamics can have laws without the specific grounding provided by statistical mechanics). An example of this on our site is looking for <a href="https://cogsci.stackexchange.com/q/525/29">behaviorist interpretations of models</a> (note that decision field theory falls more into the dynamic systems approach, so it isn't a perfect example).</p>

<p>For me, the much more prominent weakness is <strong>rationality</strong>. Bayes rule is inherently rational -- humans are not; a bayesian has to <a href="https://cogsci.stackexchange.com/q/251/29">use various hacks to account for human irrationality</a>. Connectionism does not suffer from this drawback, and neither do some exotic approaches like <a href="http://mypage.iu.edu/~jbusemey/home.html" rel="nofollow noreferrer">Busemeyer</a>'s quantum cognition (I provide a sketch in <a href="https://cogsci.stackexchange.com/a/338/29">this answer</a>). If you want to see why models based on classical probability have a difficulty explaining aspects of human irrationality, take a look at <a href="http://mypage.iu.edu/~jbusemey/QD.pdf" rel="nofollow noreferrer">Busemeyer, J. R., Wang, Z., &amp; Townsend, J. T. (2006)</a>.</p>

CC BY-SA 3.0
2012-03-15T06:00:08.363Z
2012-03-15T06:00:08.363Z
2017-04-13T12:56:16.467Z
Artem Kaznatcheev
<p>Probabilistic approaches to modelling cognition are increasing in popularity and being encouraged within the field (<a href="http://web.mit.edu/cocosci/Papers/ChaterTenenbaumYuille-TICS06.pdf">Chater, Tanenbaum, &amp; Yuille, 2006</a>). </p>

<p>What are some of the arguments against or drawbacks (other than scalability/intractability) to using this approach?</p>

CC BY-SA 3.0
2012-03-16T06:55:17.600Z
This isn't really a proper answer, but you may find informative the following BBS paper criticizing Bayesian models of cognition: Jones M., Love B. (2011). Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behav. Brain Sci. 34, 169–188.
Jake Westfall
CC BY-SA 3.0
2012-05-01T15:12:16.217Z
Bowers, J. S., & Davis, C. J. (2012). [Bayesian just-so stories in psychology and neuroscience](http://www.yangzhiping.com/files/pubs/389.pdf). Psychological Bulletin, 138(3): 389–414. and a commentary on it: Griffiths, T.L., Chater, N., Norris, D., & Pouget, A. (2012). [How the Bayesians Got Their Beliefs (and What Those Beliefs Actually Are): Comment on Bowers and Davis](http://www.yangzhiping.com/files/pubs/415.pdf). Psychological Bulletin, 138(3): 415-422.
ouyang
<computational-modeling><cognitive-modeling><bayesian>
What are some of the drawbacks to probabilistic models of cognition?
Vielle
# What are some of the drawbacks to probabilistic models of cognition?

(Asked by: username_0 on 2012-03-15)

Probabilistic approaches to modelling cognition are increasing in popularity and being encouraged within the field ([Chater, Tanenbaum, & Yuille, 2006](http://web.mit.edu/cocosci/Papers/ChaterTenenbaumYuille-TICS06.pdf)).

What are some of the arguments against or drawbacks (other than scalability/intractability) to using this approach?

**Tags:** computational-modeling, cognitive-modeling, bayesian

> **Comments:**
> **username_1**: This isn't really a proper answer, but you may find informative the following BBS paper criticizing Bayesian models of cognition: Jones M., Love B. (2011). Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behav. Brain Sci. 34, 169–188.  
> **username_2**: Bowers, J. S., & Davis, C. J. (2012). [Bayesian just-so stories in psychology and neuroscience](http://www.yangzhiping.com/files/pubs/389.pdf). Psychological Bulletin, 138(3): 389–414. and a commentary on it: Griffiths, T.L., Chater, N., Norris, D., & Pouget, A. (2012). [How the Bayesians Got Their Beliefs (and What Those Beliefs Actually Are): Comment on Bowers and Davis](http://www.yangzhiping.com/files/pubs/415.pdf). Psychological Bulletin, 138(3): 415-422.  

---

## 3 answers

### Answer by username_5 on 2012-03-15. Score: 22 | Accepted answer

Probabilistic approaches of this sort are usually referred to more specifically as the **bayesian** approach and Chater and Tanenbaum are definitely bayesians (I have not read much by Yuille and can't comment). [Bayesianism](http://lesswrong.com/lw/1to/what_is_bayesianism/) is more than just increasing in popularity and being encouraged; it is considered one of the big-4 approaches to cognitive-modeling, with the other 3 being: connectionism, rule-based, and dynamic systems. The bayesian approach has many positives and produced many great results, but since your question is about the drawbacks I will focus exclusively on that. Two major drawbacks are: neural grounding and rationality.

**Neural grounding** is a weakness that plagues all of the big-4 and cognitive science in general. The idea is that as we build models of the mind, we want to eventually [ground them in the brain](https://cogsci.stackexchange.com/q/90/29); this is a standard feature of reductionism. The bayesian approach is often summarized as "probabilities over rules", and suffers from the same difficulty of neural grounding as the rule-based approach did. It is often not clear how the brain implements this sophisticated bayesian inference (but the field is well aware of this problem, and works hard to resolve it). Is this a game killer? Not really, connectionism is often considered the more 'biologically-plausible' alternative, but most popular connectionist models can be just as [easily questioned on their biological viability](https://cogsci.stackexchange.com/q/214/29). The issue can also be sidestepped completely by saying that we do wish to address cognition at a different level that biological implementation (sort of how thermodynamics can have laws without the specific grounding provided by statistical mechanics). An example of this on our site is looking for [behaviorist interpretations of models](https://cogsci.stackexchange.com/q/525/29) (note that decision field theory falls more into the dynamic systems approach, so it isn't a perfect example).

For me, the much more prominent weakness is **rationality**. Bayes rule is inherently rational -- humans are not; a bayesian has to [use various hacks to account for human irrationality](https://cogsci.stackexchange.com/q/251/29). Connectionism does not suffer from this drawback, and neither do some exotic approaches like [Busemeyer](http://mypage.iu.edu/~jbusemey/home.html)'s quantum cognition (I provide a sketch in [this answer](https://cogsci.stackexchange.com/a/338/29)). If you want to see why models based on classical probability have a difficulty explaining aspects of human irrationality, take a look at [Busemeyer, J. R., Wang, Z., & Townsend, J. T. (2006)](http://mypage.iu.edu/~jbusemey/QD.pdf).


---

### Answer by username_3 on 2012-03-15. Score: 13

Artem gave a very good answer, but I want to add one more weaknesses of probabilistic/Bayesian models: they are not mechanistic. This is related to Artem's point about neural grounding, but is a little different. The issue is that probabilistic models don't really provide insight into the underlying mechanism that produces the observed behavior -- if you ask the question "why does it work?", the Bayesian model's answer is "because cognition is Bayesian" and that's it. Models that allow structure (and behavior) to emerge from interactions of lower-level elements provide more insight into cognitive mechanisms (though not necessarily neural mechanisms). I think (at least some) connectionist and dynamic systems models try to do this. To read more about this, check out:

McClelland, J. L., Botvinick, M. M., Noelle, D. C., Plaut, D. C., Rogers, T.T., Seidenberg, M. S., and Smith, L. B. (2010). [Letting Structure Emerge: Connectionist and Dynamical Systems Approaches to Understanding Cognition](http://www-psych.stanford.edu/~jlm/papers/McCEtAl10TiCS_LettingStructureEmerge.pdf). *Trends in Cognitive Sciences*, 14, 348-356.

> **Comments:**
> **username_4**: Griffiths et al in the same issue is a good counterpart to this article: http://www.psychology.adelaide.edu.au/personalpages/staff/amyperfors/papers/griffithsetal10tics-probabilisticmodelsofcognition.pdf  
> **username_5**: This is a good point! Although I suspect a bayesian would respond that it is not clear how connectionists or dynamic explanations are more mechanistic, since they tend to hide things in poorly understood 'emergence'.  
> **username_0**: I have had similar sentiments, one of the reasons why I asked my question.  

---

### Answer by username_6 on 2012-04-03. Score: 10

This should perhaps be a comment, but I don't have the reputation. The other two answers mention that a major drawback to the Bayesian approach is its lack of biological plausibility. However, see for instance:

> [Bayesian inference with probabilistic population codes](http://www.nature.com/neuro/journal/v9/n11/abs/nn1790.html)
>
> Ma, W.J. and Beck, J.M. and Latham, P.E. and Pouget, A.
>
> *Nature Neuroscience*, 2006

The authors propose a method by which populations of neurons may actually represent probability distributions. I don't know how convincing I find the construction, but it may be worth looking into if you're curious about the subject.

It may also be worth mentioning that while neuroscientists do seem to use the word "Bayesian" to refer generically to probabilistic approaches to reasoning, "Bayesian statistics" or "[Bayesian probability](http://en.wikipedia.org/wiki/Bayesian_probability#Objective_and_subjective_Bayesian_probabilities)" is a bit more than Bayes' rule. By itself, Bayes' rule is [just a mathematical identity in probability theory](http://en.wikipedia.org/wiki/Bayes%27_theorem#Statement_and_interpretation).

> **Comments:**
> **username_5**: I agree that this is more a comment than an answer, but it could be closer to an answer on [this question](http://cogsci.stackexchange.com/q/644/29). Also, your last paragraph is incredibly true, and I wish more researchers were conscious of this :D.  

---

