  BACKGROUND: Random-effects meta-analysis is commonly performed by first
deriving an estimate of the between-study variation, the heterogeneity, and
subsequently using this as the basis for combining results, i.e., for
estimating the effect, the figure of primary interest. The heterogeneity
variance estimate however is commonly associated with substantial uncertainty,
especially in contexts where there are only few studies available, such as in
small populations and rare diseases.
  METHODS: Confidence intervals and tests for the effect may be constructed via
a simple normal approximation, or via a Student-t distribution, using the
Hartung-Knapp-Sidik-Jonkman (HKSJ) approach, which additionally uses a refined
estimator of variance of the effect estimator. The modified Knapp-Hartung
method (mKH) applies an ad hoc correction and has been proposed to prevent
counterintuitive effects and to yield more conservative inference. We performed
a simulation study to investigate the behaviour of the standard HKSJ and
modified mKH procedures in a range of circumstances, with a focus on the common
case of meta-analysis based on only a few studies.
  RESULTS: The standard HKSJ procedure works well when the treatment effect
estimates to be combined are of comparable precision, but nominal error levels
are exceeded when standard errors vary considerably between studies (e.g. due
to variations in study size). Application of the modification on the other hand
yields more conservative results with error rates closer to the nominal level.
Differences are most pronounced in the common case of few studies of varying
size or precision.
  CONCLUSIONS: Use of the modified mKH procedure is recommended, especially
when only a few studies contribute to the meta-analysis and the involved
studies' precisions (standard errors) vary.
