Nature magazine has some tough questions to answer after it let its Wikipedia fetish get the better of its responsibilities to reporting science. The Encyclopedia Britannica has published a devastating response to Nature's December comparison of Wikipedia and Britannica, and accuses the journal of misrepresenting its own evidence.
Where the evidence didn't fit, says Britannica, Nature's news team just made it up. Britannica has called on the journal to repudiate the report, which was put together by its news team.
Independent experts were sent 50 unattributed articles from both Wikipedia and Britannica, and the journal claimed that Britannica turned up 123 "errors" to Wikipedia's 162.
But Nature sent only misleading fragments of some Britannica articles to the reviewers, sent extracts of the children's version and Britannica's "book of the year" to others, and in one case, simply stitched together bits from different articles and inserted its own material, passing it off as a single Britannica entry.
Nice "Mash-Up" - but bad science.
"Almost everything about the journal's investigation, from the criteria for identifying inaccuracies to the discrepancy between the article text and its headline, was wrong and misleading," says Britannica.
"Dozens of inaccuracies attributed to the Britannica were not inaccuracies at all, and a number of the articles Nature examined were not even in the Encyclopedia Britannica. The study was so poorly carried out and its findings so error-laden that it was completely without merit."
In one case, for example. Nature's peer reviewer was sent only the 350 word introduction to a 6,000 word Britannica article on lipids - which was criticized for containing omissions.
A pattern also emerges which raises questions about the choice of the domain experts picked by Nature's journalists.
Several got their facts wrong, and in many other cases, simply offered differences of opinion.
"Dozens of the so-called inaccuracies they attributed to us were nothing of the kind; they were the result of reviewers expressing opinions that differed from ours about what should be included in an encyclopedia article. In these cases Britannica's coverage was actually sound."
Nature only published a summary of the errors its experts found some time after the initial story, and has yet to disclose all the reviewer's notes.
So how could a respected science publication make such a grave series of errors?
1913 advertisement for the Britannica
''Encyclopædia Britannica'' has strongly criticised the scientific journal ''Nature'' for last year's investigation into the encyclopedia's accuracy compared to that of the open-source online encyclopedia, Wikipedia. ''Britannica'' also called for a retraction of the article. ''Nature'' has rejected the accusations, saying that their comparison was fair and that they do not intend to retract their report.

In a 20-page response (titled "Fatally Flawed") to the report's findings that there was not a significant difference in accuracy between the two encyclopædias, Britannica wrote that 'Almost everything about the journal's investigation ... was wrong and misleading.' The document goes on to call the investigation 'invalid', 'completely without merit' and 'careless'. Britannica also accused ''Nature'' of 'misrepresenting its own results'. The last 12 pages of this comprehensive rebuttal are responses to specific article criticisms, mainly consisting of differences of opinion on style and article composition, but some citing outright factual errors on ''Nature'''s part.

In their response, ''Nature'' refused to reveal their original data for comparison, and commented that ''Britannica'' had raised their grievances in private some time ago, and then received no further correspondence until ''Britannica'''s open letter of 22 March 2006. ''Nature'' notes that ''Britannica'' criticised 'less than half the points their reviewers raised', and states that the two encyclopædias were subject to the same treatment.

Wikipedia has been stigmatised as unreliable due to its open-source nature and a perceived problem with vandalism.  The December 2005 study was seen at the time as a major boost to the credibility of Wikipedia.  Jimmy Wales was notified by one of his team members, on 27 January 2006 that all corrections provided by ''Nature'' were in place.  Recent changes to some of the Wikipedia editing policies have attempted to resolve some of the reliability issues raised.

Wikipedia has yet to comment.