While most political scientists would agree that replications and cross-checks advance knowledge, many academic journals hesitate to publish replications. John Ishiyama, University of North Texas, discusses how replication studies can get published to create more incentives to do replications.
In the recent symposium on reproducibility in PS: Political Science & Politics (Vol 47, Issue 1), John Ishiyama, University of North Texas, discusses possible ways in which replication studies can be published.
So far, with many journals emphasizing original research articles, there are “little obvious rewards” to conduct replications in political science.
Ishiyama defends decisions of journals, such as the American Political Science Review (APSR), to focus on original work and reject replications.
[A]s editors of the APSR we are also reluctant to publish such studies in the Review, because this would open up a “cheap” way for authors to have their work published in the APSR, and every Tom, Dick, and Harriet (pardon the expression) could potentially seek to replicate some study, just to get published in the Review.
How to publish replication studies
- Occasional “Forums” in all journals, including APSR, which potentially allow for the incorporation of replication studies
- A special section in a journal which is dedicated to the production of replication reports
- Publication of replication studies in journals that are “not a general research journal“, such as PS, and Perspectives on Politics, International Studies Perspectives, and the International Studies Review
- An electronic “blog like” venue for the publication of replication studies, something like the Monkey Cage
- Publishing replication studies by major journals in an online supplement directly linked to the original articles; journals should then highlight those articles that have been replicated multiple times
- Creating a new journal for replications
The American Political Science Association must act
Ishiyama stresses that the American Political Science Association should take up this issue (and it already has), to create incentives for scholars to engage in replication studies.
[T]he APSA should consider potential alternative venues for the publication of replication studies (or perhaps “forums” or debates) of pieces that appear in APSA journals. Now it is not exactly clear how this should be done, if it could be done online, if it requires an editorial team, what the relationship would be with the existing APSA journals, and how would this be related to Cambridge University Press, but if we are tomove forward as a discipline, we must have some venue available for the publication (or at least making public) such studies.
John Ishiyama (2014). Replication, Research Transparency, and Journal Publications: Individualism, Community Models, and the Future of Replication Studies. PS: Political Science & Politics, 47, pp 78-83. doi:10.1017/S1049096513001765.
Abstract: Recently, the importance of research transparency via replication studies has been greatly discussed in most of the social sciences, political science included. Indeed, as Gherghina and Katsanidou (2013) and Freese (2007) note, to some extent, the discussion has been prompted by the tremendous changes in publishing in the past decade or so. With the enormous expansion in data availability and instant publication made possible by the Internet, there now are many opportunities to verify the findings presented in the discipline’s major journals. “Replication, replication” has not only become the mantra for political science, but for economics, psychology, and quantitative sociology as well. These developments opened a debate on how to best “guard the high standards or research practice and allow for the maximum use of current knowledge for the further development of science” (Gherghina and Katsanidou 2013, 1; for similar sentiments see King 1995).