  P values or risk ratios from multiple, independent studies, observational or
randomized, can be computationally combined to provide an overall assessment of
a research question in meta-analysis. There is a need to examine the
reliability of these methods of combination. It is typical in observational
studies to statistically test many questions and not correct the analysis
results for multiple testing or multiple modeling, MTMM. The same problem can
happen for randomized, experimental trials. There is the additional problem
that some of the base studies may be using fabricated or fraudulent data. If
there is no attention to MTMM or fraud in the base studies, there is no
guarantee that the results to be combined are unbiased, the key requirement for
the valid combining of results. We note that methods of combination are not
robust; even one extreme base study value can overwhelm standard methods of
combination. It is possible that multiple, extreme (MTMM or fraudulent) results
can feed from the base studies to bias the combined result. A meta-analysis of
observational (or even randomized studies) may not be reliable. Examples are
given along with some methods to evaluate existing base studies and
meta-analysis studies.
