By Susan Hodges
Carolyn Keefe has pointed out the importance of getting
your information from authoritative sources. In addition, we also need
to develop the ability to assess the validity and meaning of studies
and information even when they are published in respected journals.
In the Summer 2002 Citizens for Midwifery News
we mentioned publicity about a study using Washington State birth
statistics that purported to show that babies born at home are more
likely to die than if they are born in the hospital. At the time we
could only guess about flaws in the methodology.
The study was published in the August 2002 issue of the Journal of Obstetrics & Gynecology:
"Outcomes of Planned Home Births in Washington State: 1989-1996" by
Pang, Heffelfinger, Huang, Benedetti & Weiss. You can read a press
release by the American College of Obstetricians and Gynecologists
(ACOG) on their website: Home Births Double Risk of Newborn Death. For a copy of the original article, click on "August Issue of
Obstetrics & Gynecology" in the text; those words link to a PDF
file of the paper.
Now CfM and several other organizations and individuals
have critiqued the studies (see end of article for links). This study
provides a good example of the necessity for reading original papers,
and not just accepting what is said in news coverage. It is also
necessary to read any "scientific"Â paper critically, because
medical and scientific journals do sometimes publish papers of dubious
When I finally got my hands on a copy of the Pang
study, I read it with pencil in hand to note all my questions. I kept
in mind what I know about home birth, and especially aspects of home
birth about which the researchers (all of whom are medical doctors)
were likely to be ignorant and which might affect the data and
conclusions. I looked for statements that were based on assumptions,
rather than data, and I questioned the assumptions. More difficult, but
also important, is looking for what is NOT included in the study. It is
more important to ask the questions than to answer them; asking
questions will help you think about the study and what it actually
shows and doesn't show.
Here are some of the kinds of questions one can ask when reading this type of paper:
- Did the authors refer to and discuss all or most of the published articles on the same or related topic?
- What is the stated objective of the study? Does the study or experiment actually answer this question?
- Read the materials and methods section carefully.
What is the source of the data and is it of good quality? (Has it been
verified? How inaccurate can it be?) Are the data categories adequately
defined so you can tell exactly what is included and what is not? How
have the authors manipulated the data? Have they made assumptions? Have
they justified their assumptions and shown that they are reasonable to
make? Do you think their assumptions are reasonable and valid for this
study? Why or why not?
- When data from two or more groups are compared,
have the authors demonstrated that the groups, and the data from each,
are essentially the same except for the aspects being compared?
- Do the results (what was found out) make sense?
Are the numbers large enough so that inaccuracies in the data
collection will not affect the statistical significance of the results?
- The discussion (or conclusions) should include
the authors' interpretations of their results. What do the results
mean? What can and cannot be concluded? What are the strengths and
shortcomings of the methodology? What problems occurred and could those
problems affect the results? How do the results compare with the
results of other related or similar studies? How are similarities or
differences in results explained?
This study on Washington State home births has many
flaws and problems, many of which become obvious when read critically
(i.e., questioning everything). Without going into great detail, here
is an example. The title reads: "Outcomes of Planned Home Births in
Washington State: 1989-1996," but the authors write in the Materials
and Methods section that "Because Washington State birth certificates
do not identify which home births were planned, we defined planned home
births." This sentence alerts us to the fact that the authors were
using inadequate data, and we had better look very carefully at the
assumptions they used to decide which home births were "planned."
Here is what they tell us they included:
- "singleton newborns" - okay, since this helps to make the home and hospital groups equivalent.
- "of at least 34 weeks' gestation - not
okay since babies born "at home" at 34-37 weeks are likely to be
unplanned, precipitous births, at higher risk due to prematurity.
Experienced home birth midwives would not plan to attend at home at
less than 37 weeks so these should not be included (although later in
the paper they claim that omitting births prior to 37 weeks did not
affect the relative rates of neonatal mortality).
- "who were delivered at home - maybe OK,
but we do not know how this is designated on the birth certificates,
and whether or not it includes unplanned out-of-hospital births (in the
car, on the way, etc.).
- "and who had a midwife, nurse or physician
listed as either the birth attendant or certifier on the birth
certificate (if an attendant is not listed on the birth certificate,
then the person listed as the certifier attended the delivery)" - we
know that those babies born "on the way" would then go to the
hospital where someone would sign the birth certificate - NOT a
planned home birth.
We don't even have to go any further in the study to
be certain that at least some unplanned, unattended, even premature
births will have been incorrectly included in the cohort of "planned
home births." Any neonatal deaths, postpartum bleeding, need for
resuscitation for more than 30 minutes (all outcomes included in the
study) associated with these high risk births would be wrongly
attributed to the "planned home birth." Even the authors
acknowledge later in the paper that this is a problem that could skew
the results to make "planned home birth" appear more dangerous.
The authors chose to look primarily at neonatal deaths,
even though the rate is very low for healthy normal pregnant women
regardless of where they give birth. The study reports neonatal
mortality rates of 1.5/1000 (hospital) vs. 3.5/1000 (home). The study
offers no basis for knowing the magnitude of misclassifications of
births in the "planned home birth" category; it is possible that
eliminating wrongly classified births would diminish the difference to
the point of statistical insignificance.
What about information
and data that are not included but should be? For example, there is no
mention made of unpreventable neonatal deaths, i.e., when the cause of
death was unrelated to the intended or actual site of birth (congenital
anomalies incompatible with life, for example). In addition, we cannot
know from this paper how many of the neonatal deaths occurred in the
unknown number of unplanned/unattended "home births" that were
wrongly included as "planned home births." Addressing another
outcome, the authors claim that women "intending a home delivery were
more likely to have prolonged labor." However, we know that there can
be substantial differences between home and hospital regarding when the
beginning of labor is counted, and that typical hospital protocols call
for interventions to speed or augment labor, so one might anticipate
that hospital births would be less likely to have "prolonged
labor," yet this is not discussed. In fact, medical interventions
that impact on outcomes for both mother and baby, such as episiotomy,
epidural and other drugs, induction, augmentation of labor, forceps,
vacuum extraction and c-section, are not even mentioned, let alone
discussed in relation to this study.
Finally, even if the findings of the study were
"true," not even the authors claim anything more than an
observation of an association in this set of data shaped by their
assumptions. In other words, this study in no way was designed to show,
and does not show, a causal relationship. The results cannot
legitimately be used to predict even the risk of neonatal death as a
result of planning a home birth. One should also note that while the
original study, with all its flaws, at least has a neutral title, the
ACOG press release overstates the study's findings with an
inflammatory and misleading headline, and fails to even mention the
shortcomings and limitations of the study.
These are just a few of the flaws that can be found by
simply reading the study with a critical eye. However, while the Pang
study provides a good exercise for critical reading, you do not have to
figure it all out for yourself! CfM has prepared two fliers, one with
the key criticisms of the study and the other quoting the authors'
own words, in the study, regarding its problems and limitations. In
addition excellent and thorough critiques can be found on-line (see
If this study comes up in your newspaper, or among
legislators, use these resources to show up the flaws of the study. The
CfM fliers in particular are designed with press and state officials in
On-Line critiques of the Washington State planned home birth study by Pang et al:
Citizens for Midwifery fact sheets cover the main points of criticism.
Henci Goer, author of The Thinking Woman's Guide to a Better Birth, has written a critique "Homebirth: Is it really a safe option?" that is posted on ParentsPlace.com (where she is the Birth Guru). Click on "Birth Guru" under Ask The Expert, scroll down to "articles by subject."
MAWS (Midwives Association of Washington State) has written a response. Their statement includes a good list of references.
Faith Gibson has also written a detailed commentary and critique.
For an excellent and more thorough look at the topic of
home birth safety than the Pang et al. study, using much higher quality
and more detailed data, more births, and logistic regression analyses
to show interactions of risk factors, see Peter Schlenzka's
dissertation The Safety of Alternative Approaches to Childbirth - located toward the bottom of the page.
Reprinted from Citizens for Midwifery News, Fall 2002. Permission to reprint with attribution.