  We are concerned with testing replicability hypotheses for many endpoints
simultaneously. This constitutes a multiple test problem with composite null
hypotheses. Traditional $p$-values, which are computed under least favourable
parameter configurations, are over-conservative in the case of composite null
hypotheses. As demonstrated in prior work, this poses severe challenges in the
multiple testing context, especially when one goal of the statistical analysis
is to estimate the proportion $\pi_0$ of true null hypotheses. Randomized
$p$-values have been proposed to remedy this issue. In the present work, we
discuss the application of randomized $p$-values in replicability analysis. In
particular, we introduce a general class of statistical models for which valid,
randomized $p$-values can be calculated easily. By means of computer
simulations, we demonstrate that their usage typically leads to a much more
accurate estimation of $\pi_0$. Finally, we apply our proposed methodology to a
real data example from genomics.
