  Replication studies are essential for assessing the credibility of claims
from original studies. A critical aspect of designing replication studies is
determining their sample size; a too small sample size may lead to inconclusive
studies whereas a too large sample size may waste resources that could be
allocated better in other studies. Here, we show how Bayesian approaches can be
used for tackling this problem. The Bayesian framework allows researchers to
combine the original data and external knowledge in a design prior distribution
for the underlying parameters. Based on a design prior, predictions about the
replication data can be made, and the replication sample size can be chosen to
ensure a sufficiently high probability of replication success. Replication
success may be defined by Bayesian or non-Bayesian criteria, and different
criteria may also be combined to meet distinct stakeholders and enable
conclusive inferences based on multiple analysis approaches. We investigate
sample size determination in the normal-normal hierarchical model where
analytical results are available and traditional sample size determination is a
special case where the uncertainty on parameter values is not accounted for. We
use data from a multisite replication project of social-behavioral experiments
to illustrate how Bayesian approaches can help design informative and
cost-effective replication studies. Our methods can be used through the R
package BayesRepDesign.
