Two disturbing stories this week demonstrated the dangers of rejecting best practice of systematic review where the literature on a subject is surveyed methodically to find all the evidence.
Firstly, the US Public Library of Science used a court order to obtain evidence showing how the pharmaceutical company Wyeth employed commercial "ghost writers" to produce reviews, published in academic journals, under the names of academic authors.
It is claimed that these articles, published between 1998 and 2005, stressed the benefits of taking hormones to protect against problems such as heart disease, dementia, and ageing skin, while playing down the risks.
To understand the full damage that these tainted reviews can do, we need to understand a little about the structure of academic knowledge.
In a formal academic paper every claim is referenced to another academic paper: either original research describing a piece of primary research or a review which summarises an area.
This gives us an opportunity to study how ideas spread, and myths grow, because in theory you could trace who references what, and how, to see a belief system evolve from the original data.
Secondly, a seminal article was published this month in the British Medical Journal where Steven Greenberg from Harvard medical school focused on an arbitrary hypothesis. His case study was the idea that a protein called beta amyloid is produced in the skeletal muscle of patients who have a condition called inclusion body myositis (IBM).
Hundreds of papers have been written on this, with thousands of citations, and using network theory Greenberg produced a map demonstrating who cited what. By looking at this network he could identify intersections with the most traffic. All of the 10 most influential papers expressed the view that beta amyloid is produced in the muscle of patients with IBM. In reality, this is not supported by the evidence. So how did this situation arise?
Four lab papers did find beta amyloid in IBM patients' muscle tissue, and these were among the top 10 most influential papers. But there were also six papers which contradicted the hypothesis. Crucially, they were ignored.
Using the interlocking web of citations you can see how this happened. A small number of review papers funnelled large amounts of traffic through the network. These acted like a lens, collecting and focusing citations on the papers supporting the hypothesis.
But Greenberg goes beyond documenting bias. By studying the network he showed how these reviews exerted influence beyond their own readerships. He also showed that some papers did cite contradictory research but distorted it.
One paper reported no beta amyloid in three of five patients with IBM, and its presence in only a "few fibres" in the remaining two patients; but three subsequent papers cited this data, saying that it "confirmed" the hypothesis.
This is an exaggeration at best, but the power of the social network theory approach is to show what happened next: over the following 10 years these three supportive citations were the root of 7,848 supportive citation paths, producing chains of false claim in the network, amplifying the distortion. This is the story of how myths and misapprehensions arise. It also shows why systematic reviews are important, and that's why ghostwriting should be stopped.