  The low replication rate of published studies has long concerned the social
science community, making understanding the replicability a critical problem.
Several studies have shown that relevant research communities can make
predictions about the replicability of individual studies with above-chance
accuracy. Follow-up work further indicates that laypeople can also achieve
above-chance accuracy in predicting replicability when experts interpret the
studies into short descriptions that are more accessible for laypeople. The
involvement of scarce expert resources may make these methods expensive from
financial and time perspectives. In this work, we explored whether laypeople
can predict the replicability of social science studies without expert
intervention. We presented laypeople with raw materials truncated from
published social science papers and elicited their answers to questions related
to the paper. Our results suggested that laypeople were engaged in this
technical task, providing reasonable and self-contained answers. The majority
of them also demonstrated a good understanding of the material. However, the
solicited information had limited predictive power on the actual replication
outcomes. We further discuss several lessons we learned compared to the
approach with expert intervention to inspire future works.
