The story suffers from a lack of independent voices and a lack of clear analysis of the data. There was little more than a nod to the potential harms done in over-screening. Detection of a marker for a disease is not the same as preventing a disease or preventing a death.
The story says that “very few people avail themselves of [screening] colonoscopy – only about 20 percent.” That was one researcher’s estimate – someone with a vested financial interest in this research. But the fact is that the CDC BRFSS survey estimates that about 60% of US adults are up to date with screening, most with colonoscopy. That is a big difference from 20%, and one has to wonder why that was not double-checked. Still, there clearly are a lot of people being missed. How to improve those numbers is up for debate. Stories about new types of screening need to make clear comparisons between the success rates of colonoscopies now and the possible success rates of these emerging technologies.
This story states that “Though the cost has not been set, it could be as low as $300 per test.” But why take that cheerleading stance about how LOW it might be? And who says it may be that low? (A Reuters story attributed costs estimates to the company.) Who’s the source? Why not tell us that it could be AS HIGH AS $xxx per test? Nonethless, we give it a satisfactory score for giving some ballpark estimate.
The story does quantify the potential benefits, but it doesn’t put this into the context of the performance of other existing colon cancer screening tests – including other stool tests. So the reader gets no sense of the scope of the potential benefit.
No mention of potential harms. If “most people using the new test would have negative results” and more people are going to be likely to take a test that is non-invasive doesn’t that increase the chances of missed cancers? And, conversely, if more people are screened and more tumors are found doesn’t that increase the chances of unnecessary surgical interventions? The New York Times story, by comparison, addressed false positives.
This is where the story falls down on the job. There is a lot of information packed into this story but not a lot of clarity. It presents the findings as “The Holy Grail” appearing to get close. But a significant limitation that is only indirectly noted is that initial accuracy studies in non-representative samples of cases and non-cases (like this study) usually overestimate performance. That is why they are planning another study. Health Day did a better job by including the quote from Dr. Durado Brooks of the American Cancer Society: “Showing that in a small group of samples is very different from demonstrating that in a population where only a small number of individuals are going to have polyps of that size. Then we will know if this is a big step forward.” WebMD quoted Brooks: “The performance may decrease considerably when the test is used in a large population of heatlhy individuals.”
No disease mongering here. The story says, “The United States has about 150,000 new diagnoses and 50,000 colon cancer deaths each year. The disease kills about 40 percent who are diagnosed, reflecting the fact that most people do not undergo colonoscopy until they have symptoms such as bleeding – at which point the cancer may be advanced.”
The story does not have a single source who is not connected to the technology or the company in some way. It also describes Johns Hopkins University researcher Bert Vogelstein this way. “He is not involved in Exact Science’s research, but the company licensed some of his technology from Hopkins for the test.” The New York Times was more straightforward and more informative. It said, “a cancer expert at Johns Hopkins University who is also an adviser to the company.” Conflicts of interest are noted throughout, but disclosure can’t beat independence for helping readers understand the importance of this research.
The story did not compare the results to existing colon cancer screening tests or compare the results with those of another new test being developed by Epigenomics, as the New York Times did.
The quote “Now, the gold standard for screening is a colonoscopy, which requires a colon clean-out only slightly less unpleasant than stomach flu, followed by threading a scope through the intestines under anesthesia” is misleading in three ways. First, colonoscopy is not the “gold standard.” It is simply one option. Two, the preparation for colonoscopy is not similar to stomach flu, at least for most people. And three, patients don’t receive “anesthesia” at least as defined in the way most people think about that word, “general anesthesia.”
The story makes it clear that the test is not available currently. It says, “The DNA test is still experimental, hasn’t been validated under real-life conditions, and will take at least another year of development, he said.”
The story talks about this being a “novel test” that screens for a “novel marker.” This appears to be true, but the story fails to discuss a competing screening test also in development. We think that the story should have included at least a mention of the other new test in development as the New York Times did.
The story does not rely solely on a news release.