<p>This is complicated.  There's no easy answer, but the outlook for replicability/reproducibility of a lot of the empirical evidence is not great.</p>
<p>The R-index (that the blog authors use to rank the chapters) is itself a heuristic, which shouldn't be taken as statistical evidence that a study will <strong>necessarily</strong> fail to replicate.  That being said, it could potentially be considered a useful heuristic, as is the idea that people process information over time (i.e., some mental processes happen more quickly than others).</p>
<p>An issue of <em>Psychological Inquiry</em> might be helpful to people who want to learn about the most recent conversation among <strong>a select group</strong> of psychological scientists.  This journal publishes a &quot;target article&quot; and allows other people in the field to comment, to which the original authors reply.  The whole conversation is published as an issue - this one is from April 2021.  Without going into detail, many priming effects do not replicate, and social priming might be a terrible term to use, in part because it is associated with questionable scientific literature (<a href="https://www.researchgate.net/profile/Jeffrey-Sherman-3/publication/350572142_There%27s_Nothing_Social_about_Social_Priming_Derailing_the_Train_Wreck/links/6074a7a092851cb4a9d80c2b/Theres-Nothing-Social-about-Social-Priming-Derailing-the-Train-Wreck.pdf" rel="noreferrer">target article here</a>, <a href="http://uweb.cas.usf.edu/%7Edrohrer/pdfs/Harris_et_al_2021PI.pdf" rel="noreferrer">one cherry-picked commentary here</a>, <a href="https://www.researchgate.net/profile/Jeffrey-Sherman-3/publication/350568845_A_Final_Word_on_Train_Wrecks/links/6074a780299bf1f56d51ba47/A-Final-Word-on-Train-Wrecks.pdf" rel="noreferrer">the authors of the target article reply to the commentaries here</a>).</p>
<p>Kahneman himself warned of problems with the literature in 2012 (see a scientific news report <a href="https://www.nature.com/articles/nature.2012.11535" rel="noreferrer">here</a> (4) and his email <a href="https://www.nature.com/news/polopoly_fs/7.6716.1349271308!/suppinfoFile/Kahneman%20Letter.pdf" rel="noreferrer">here</a>).</p>
<ol>
<li><p>Sherman, J. W., &amp; Rivers, A. M. (2021). There’s nothing social about social priming: derailing the “train wreck”. Psychological Inquiry, 32(1), 1-11.</p>
</li>
<li><p>Harris, C., Rohrer, D., &amp; Pashler, H. (2021). A train wreck by any other name. Psychological Inquiry, 32(1), 17-23.</p>
</li>
<li><p>Sherman, J. W., &amp; Rivers, A. M. (2021). A final word on train wrecks. Psychological Inquiry, 32(1), 49-52.</p>
</li>
<li><p>Yong, E. (2012). Nobel laureate challenges psychologists to clean up their act. Nature News.</p>
</li>
</ol>
<p>Hope this helps.</p>

CC BY-SA 4.0
2021-07-01T07:13:41.987Z
Thanks, yes, it is helpful. I've read that the priming research was a mess, but priming studies are only like 15% of the studies used in the book. This means some 40+% of studies on effects other than priming are expected not to replicate as per this statistical tool. I was wandering about those other effects too.
user2628084
CC BY-SA 4.0
2021-07-05T02:44:40.633Z
True, there was a lot in that book.  Is there a specific topic you were curious about?  They list numerous chapters that received a "failing grade" in the R-index blog post you mentioned (low R-index means that replications are heuristically less likely to produce similar results).
pep
CC BY-SA 4.0
2021-07-01T01:33:15.047Z
2021-07-01T01:33:15.047Z
pep
<blockquote>
<p>'So, replicability [of all studies in this book] is somewhere between 12% and 46%. Even if half of the results are replicable, we do not know which results are replicable and which one’s are not.'</p>
</blockquote>
<blockquote>
<p>'Readers of “Thinking: Fast and Slow” should read the book as a subjective account by an eminent psychologists, rather than an objective summary of scientific evidence.'</p>
</blockquote>
<p><a href="https://replicationindex.com/2020/12/30/a-meta-scientific-perspective-on-thinking-fast-and-slow/" rel="nofollow noreferrer">A Meta-Scientific Perspective on “Thinking: Fast and Slow</a></p>
<p>I have no background in social sciences or statistics so I don't know if claims and math in this article are correct.  Could somebody with more knowledge comment on this? This is HUGE if true.</p>

CC BY-SA 4.0
2021-06-29T22:49:52.497Z
Why "huge if true"? Pop-science books aren't really how science works and the "replication crisis" in psychology is quite well-known.
Bryan Krause
CC BY-SA 4.0
2021-06-30T05:44:30.113Z
Well the book informed countless sci popularizers and countless people. I've seen this book praised by people with psych backgrounds as 'One of the very few pop-psych books that largely got the scientific literature right'. https://www.reddit.com/r/AskSocialScience/comments/5t17ry/what_do_psychologists_today_think_of_kahneman/?utm_medium=android_app&utm_source=share
user2628084
CC BY-SA 4.0
2021-06-30T10:25:16.813Z
I have expertise in social psychology and statistics. A lot of the work mentioned hasn't replicated: there is not a consensus about exactly how much. Not replicating doesn't mean not true: interpreting replications is not trivial.
Cameron Brick
<bias><statistics><behavioral-economics><replication>
Is most of Kahneman's 'Thinking fast and slow' not supported by evidence/non replicable?
user2628084
# Is most of Kahneman's 'Thinking fast and slow' not supported by evidence/non replicable?

(Asked by: username_0 on 2021-06-29)

> 'So, replicability [of all studies in this book] is somewhere between 12% and 46%. Even if half of the results are replicable, we do not know which results are replicable and which one’s are not.'

> 'Readers of “Thinking: Fast and Slow” should read the book as a subjective account by an eminent psychologists, rather than an objective summary of scientific evidence.'

[A Meta-Scientific Perspective on “Thinking: Fast and Slow](https://replicationindex.com/2020/12/30/a-meta-scientific-perspective-on-thinking-fast-and-slow/)

I have no background in social sciences or statistics so I don't know if claims and math in this article are correct. Could somebody with more knowledge comment on this? This is HUGE if true.

**Tags:** bias, statistics, behavioral-economics, replication

> **Comments:**
> **username_1**: Why "huge if true"? Pop-science books aren't really how science works and the "replication crisis" in psychology is quite well-known.  
> **username_0**: Well the book informed countless sci popularizers and countless people. I've seen this book praised by people with psych backgrounds as 'One of the very few pop-psych books that largely got the scientific literature right'. https://www.reddit.com/r/AskSocialScience/comments/5t17ry/what_do_psychologists_today_think_of_kahneman/?utm_medium=android_app&utm_source=share  
> **username_2**: I have expertise in social psychology and statistics. A lot of the work mentioned hasn't replicated: there is not a consensus about exactly how much. Not replicating doesn't mean not true: interpreting replications is not trivial.  

---

## 1 answers

### Answer by username_3 on 2021-07-01. Score: 8

This is complicated. There's no easy answer, but the outlook for replicability/reproducibility of a lot of the empirical evidence is not great.

The R-index (that the blog authors use to rank the chapters) is itself a heuristic, which shouldn't be taken as statistical evidence that a study will **necessarily** fail to replicate. That being said, it could potentially be considered a useful heuristic, as is the idea that people process information over time (i.e., some mental processes happen more quickly than others).

An issue of *Psychological Inquiry* might be helpful to people who want to learn about the most recent conversation among **a select group** of psychological scientists. This journal publishes a "target article" and allows other people in the field to comment, to which the original authors reply. The whole conversation is published as an issue - this one is from April 2021. Without going into detail, many priming effects do not replicate, and social priming might be a terrible term to use, in part because it is associated with questionable scientific literature ([target article here](https://www.researchgate.net/profile/Jeffrey-Sherman-3/publication/350572142_There%27s_Nothing_Social_about_Social_Priming_Derailing_the_Train_Wreck/links/6074a7a092851cb4a9d80c2b/Theres-Nothing-Social-about-Social-Priming-Derailing-the-Train-Wreck.pdf), [one cherry-picked commentary here](http://uweb.cas.usf.edu/%7Edrohrer/pdfs/Harris_et_al_2021PI.pdf), [the authors of the target article reply to the commentaries here](https://www.researchgate.net/profile/Jeffrey-Sherman-3/publication/350568845_A_Final_Word_on_Train_Wrecks/links/6074a780299bf1f56d51ba47/A-Final-Word-on-Train-Wrecks.pdf)).

Kahneman himself warned of problems with the literature in 2012 (see a scientific news report [here](https://www.nature.com/articles/nature.2012.11535) (4) and his email [here](https://www.nature.com/news/polopoly_fs/7.6716.1349271308!/suppinfoFile/Kahneman%20Letter.pdf)).

1. Sherman, J. W., & Rivers, A. M. (2021). There’s nothing social about social priming: derailing the “train wreck”. Psychological Inquiry, 32(1), 1-11.
2. Harris, C., Rohrer, D., & Pashler, H. (2021). A train wreck by any other name. Psychological Inquiry, 32(1), 17-23.
3. Sherman, J. W., & Rivers, A. M. (2021). A final word on train wrecks. Psychological Inquiry, 32(1), 49-52.
4. Yong, E. (2012). Nobel laureate challenges psychologists to clean up their act. Nature News.

Hope this helps.

> **Comments:**
> **username_0**: Thanks, yes, it is helpful. I've read that the priming research was a mess, but priming studies are only like 15% of the studies used in the book. This means some 40+% of studies on effects other than priming are expected not to replicate as per this statistical tool. I was wandering about those other effects too.  
> **username_3**: True, there was a lot in that book.  Is there a specific topic you were curious about?  They list numerous chapters that received a "failing grade" in the R-index blog post you mentioned (low R-index means that replications are heuristically less likely to produce similar results).  

---

