Jonah Lehrer recently wrote one of the most disturbing articles - as in 'rock-your-world/I-didn't-know-that' kind of 'disturbing' - for The New Yorker. His article is called The Truth Wears Off, and in it Lehrer points out that science isn't, em, working the way it's supposed to. He reports example after example of studies (e.g.: drug tests) that have been repeated over time - only for the statistically significant effect reported in the earlier studies to gradually disappear in the later studies. There's the freaky part: the effects gradually disappear. If it was all down to mere randomness then the effects might fail to appear in one set of tests and then suddenly appear (even more strongly perhaps) in a subsequent set of tests. But they don't. Instead they just 'wear off'.
And here's the even freakier part: it isn't just studies concerning humans that are seeing these gradual declines. For example, research in the early 1990s among different bird species showed that female birds were more likely to mate with male birds with symmetrical feathers. But then something strange happened:
In 1994, there were fourteen published tests of symmetry and sexual selection, and only eight found a correlation. In 1995, there were eight papers on the subject, and only four got a positive result. By 1998, when there were twelve additional investigations of fluctuating asymmetry, only a third of them confirmed the theory. Worse still, even the studies that yielded some positive result showed a steadily declining effect size. Between 1992 and 1997, the average effect size shrank by eighty per cent.Perhaps the female birds were reading the journal articles and decided to not be so predictable?
But at least scientists now have a name for this: they call it 'the decline effect'. Though as Professor Jonathan Schooler points out in a relate article in Nature, just because you give something a name doesn't mean you have an explanation for it! Both Schooler and Lehrer feature in a brilliant Radiolab podcast on Cosmic Habituation - their phrase for 'the decline effect'. Sounds even better.
One obvious message from these observations is to proceed cautiously with claims about scientific validity and certainly with claims about the potency of new drugs. Even existing ones - as noted in my recent post on mental illness in Ireland. Indeed, recent research on Prozac has shown that placebos are 80% as effective as 'the real thing'.
We're only scratching the surface of the placebo effect and related mysteries like the decline effect. But we don't have to wait until the science is sorted. Perhaps as part of a budget balancing experiment we should try putting the entire nation on placebos for a while just to see what difference it makes. The outcome might be surprising...
ht Barking Up The Wrong Tree