<p>It's not a journal article, but it's a project, called Reproducibility Project.
They are re-doing psychological studies to estimate the reproducibility of a sample of studies. They try to stay as close as possible on the original study (they e-mail the author(s) for the exact stimuli etc.).
<a href="http://openscienceframework.org/" rel="nofollow">Here</a>'s the main site and <a href="https://docs.google.com/spreadsheet/ccc?key=0AvIo2znxWnxZdERIS2xqNnNxUUZRRTB5LVJxckhiY3c" rel="nofollow">here</a>'s the link to their spreadsheet.</p>

<p><a href="http://go.nature.com/9gcjrg" rel="nofollow">This</a> is a short article promoting the sharing of data in psychology. Another related study (by the same author) is also mentioned.</p>

CC BY-SA 3.0
2012-05-17T21:41:53.300Z
It is PLoS (Public Library of Science) and it is not a journal, but it publishes [several journals](http://www.plos.org/publications/journals/) that have names that start with PLoS. None of these journals are psychology specific. Further, these journals don't "claim to be 'open-access'". They basically defined what it means to be open-access. However, I have not heard of them trying to promote reproducible data analysis, especially in psychology (they concentrate on bio and medicine mostly, although PLoS One does publish some other things); I don't see how your last paragraph has relevance.
Artem Kaznatcheev
CC BY-SA 3.0
2012-05-17T20:04:01.643Z
2012-06-18T21:23:44.720Z
2012-06-18T21:23:44.720Z
Mien
<p>I consider reproducible analysis to be really important.
In particular, I'd like to see more researchers in psychology:</p>

<ul>
<li>sharing their data</li>
<li>producing their journal articles using literate programming technologies like <a href="http://www.statistik.lmu.de/~leisch/Sweave/" rel="nofollow noreferrer">Sweave</a> and <a href="http://yihui.name/knitr/" rel="nofollow noreferrer">knitr</a> that combine data analysis code and text into an integrated documenst.</li>
</ul>

<p>I consider this particularly important in light of  concerns over research fraud (e.g., data fabrication) and  various forms of data fishing (e.g., testing many hypotheses and only reporting the significant one).</p>

<p>Certainly in statistical fields, there is a lot of discussion about the importance of reproducible analyses (e.g., see <a href="https://stats.stackexchange.com/questions/14999/how-are-we-defining-reproducible-research">here</a>, and <a href="https://stats.stackexchange.com/questions/1980/complete-substantive-examples-of-reproducible-research-using-r">here</a>). However, I sometimes feel that in psychology this is only starting to filter through.</p>

<p>Thus, my question is:</p>

<ul>
<li><strong>Are there any journal articles in psychology that have advocated reproducible analysis?</strong></li>
</ul>

<p>There are many angles that such articles might take: reward structures that discourage open science and reproducible analysis; technical challenges which make it difficult to do such analyses; how-to articles that outline how to perform reproducible analyses in psychology. I'd be interested in any articles that address these topics.</p>

CC BY-SA 3.0
2017-09-07T23:47:59.260Z
[This reanalysis](https://f1000research.com/articles/5-1014/v1), published in the [Reproducibility and Robustness](https://f1000research.com/gateways/PRR) gateway on *F1000Research*, undertakes to reproduce (part of) an earlier published report, in order to criticize its methods. (A fuller story is available in blog form [here](https://blog.f1000.com/2016/09/12/moving-to-opportunity-challenging-an-analysis-of-poverty-opportunity-and-ptsd/).) A special point of interest is that the reproduction had to be done in SAS! All of the reanalysis code is publicly available.
David C. Norris
CC BY-SA 3.0
2012-06-18T21:31:43.663Z
I don't own a copy of the APA manual myself, so I couldn't look this up, but this is a quote from a paper by Bakker & Wicherts (2011), [The (mis)reporting of statistical results in psychology journals](http://www.springerlink.com/content/g81j442645816724/): "...since standard
8.14 of the ethical standards of the American Psychological
Association (2010) states that data should be shared after
research results are published...". I thought this might interested you.
Mien
<reference-request><statistics><reproducible-research>
Are there any journal articles in psychology that have promoted and discussed reproducible data analysis?
Jeromy Anglim
# Are there any journal articles in psychology that have promoted and discussed reproducible data analysis?

(Asked by: username_0 on 2012-05-17)

I consider reproducible analysis to be really important.
In particular, I'd like to see more researchers in psychology:

* sharing their data
* producing their journal articles using literate programming technologies like [Sweave](http://www.statistik.lmu.de/~leisch/Sweave/) and [knitr](http://yihui.name/knitr/) that combine data analysis code and text into an integrated documenst.

I consider this particularly important in light of concerns over research fraud (e.g., data fabrication) and various forms of data fishing (e.g., testing many hypotheses and only reporting the significant one).

Certainly in statistical fields, there is a lot of discussion about the importance of reproducible analyses (e.g., see [here](https://stats.stackexchange.com/questions/14999/how-are-we-defining-reproducible-research), and [here](https://stats.stackexchange.com/questions/1980/complete-substantive-examples-of-reproducible-research-using-r)). However, I sometimes feel that in psychology this is only starting to filter through.

Thus, my question is:

* **Are there any journal articles in psychology that have advocated reproducible analysis?**

There are many angles that such articles might take: reward structures that discourage open science and reproducible analysis; technical challenges which make it difficult to do such analyses; how-to articles that outline how to perform reproducible analyses in psychology. I'd be interested in any articles that address these topics.

**Tags:** reference-request, statistics, reproducible-research

> **Comments:**
> **username_2**: I don't own a copy of the APA manual myself, so I couldn't look this up, but this is a quote from a paper by Bakker & Wicherts (2011), [The (mis)reporting of statistical results in psychology journals](http://www.springerlink.com/content/g81j442645816724/): "...since standard 8.14 of the ethical standards of the American Psychological Association (2010) states that data should be shared after research results are published...". I thought this might interested you.  
> **username_1**: [This reanalysis](https://f1000research.com/articles/5-1014/v1), published in the [Reproducibility and Robustness](https://f1000research.com/gateways/PRR) gateway on *F1000Research*, undertakes to reproduce (part of) an earlier published report, in order to criticize its methods. (A fuller story is available in blog form [here](https://blog.f1000.com/2016/09/12/moving-to-opportunity-challenging-an-analysis-of-poverty-opportunity-and-ptsd/).) A special point of interest is that the reproduction had to be done in SAS! All of the reanalysis code is publicly available.  

---

## 1 answers

### Answer by username_2 on 2012-05-17. Score: 9 | Accepted answer

It's not a journal article, but it's a project, called Reproducibility Project.
They are re-doing psychological studies to estimate the reproducibility of a sample of studies. They try to stay as close as possible on the original study (they e-mail the author(s) for the exact stimuli etc.).
[Here](http://openscienceframework.org/)'s the main site and [here](https://docs.google.com/spreadsheet/ccc?key=0AvIo2znxWnxZdERIS2xqNnNxUUZRRTB5LVJxckhiY3c)'s the link to their spreadsheet.

[This](http://go.nature.com/9gcjrg) is a short article promoting the sharing of data in psychology. Another related study (by the same author) is also mentioned.

> **Comments:**
> **username_3**: It is PLoS (Public Library of Science) and it is not a journal, but it publishes [several journals](http://www.plos.org/publications/journals/) that have names that start with PLoS. None of these journals are psychology specific. Further, these journals don't "claim to be 'open-access'". They basically defined what it means to be open-access. However, I have not heard of them trying to promote reproducible data analysis, especially in psychology (they concentrate on bio and medicine mostly, although PLoS One does publish some other things); I don't see how your last paragraph has relevance.  

---

