  Various methods of combining individual p-values into one p-value are widely
used in many areas of statistical applications. We say that a combining method
is valid for arbitrary dependence (VAD) if it does not require any assumption
on the dependence structure of the p-values, whereas it is valid for some
dependence (VSD) if it requires some specific, perhaps realistic but
unjustifiable, dependence structures. The trade-off between validity and
efficiency of these methods is studied via analyzing the choices of critical
values under different dependence assumptions. We introduce the notions of
independence-comonotonicity balance (IC-balance) and the price for validity. In
particular, IC-balanced methods always produce an identical critical value for
independent and perfectly positively dependent p-values, a specific type of
insensitivity to a family of dependence assumptions. We show that, among two
very general classes of merging methods commonly used in practice, the Cauchy
combination method and the Simes method are the only IC-balanced ones.
Simulation studies and a real data analysis are conducted to analyze the sizes
and powers of various combining methods in the presence of weak and strong
dependence.
