  Meta-analyses in orphan diseases and small populations generally face
particular problems including small numbers of studies, small study sizes, and
heterogeneity of results. However, the heterogeneity is difficult to estimate
if only very few studies are included. Motivated by a systematic review in
immunosuppression following liver transplantation in children we investigate
the properties of a range of commonly used frequentist and Bayesian procedures
in extensive simulation studies. Furthermore, the consequences for interval
estimation of the common treatment effect in random effects meta-analysis are
assessed. The Bayesian credibility intervals using weakly informative priors
for the between-trial heterogeneity exhibited coverage probabilities in excess
of the nominal level for a range of scenarios considered. However, they tended
to be shorter than those obtained by the Knapp-Hartung method, which were also
conservative. In contrast, methods based on normal quantiles exhibited
coverages well below the nominal levels in many scenarios. With very few
studies, the performance of the Bayesian credibility intervals is of course
sensitive to the specification of the prior for the between trial
heterogeneity. In conclusion, the use of weakly informative priors as
exemplified by half-normal priors (with scale 0.5 or 1.0) for log odds ratios
is recommended for applications in rare diseases.
