Want to get published? Say you discovered something
Published 15 September, 2009, 13:38
Scientific peer-reviewed journals are biased in choosing papers with positive results for publications, a new experiment suggests.
“No news is good news” is probably a bad piece of common wisdom for researchers seeking to get their work published. Papers with no positive result are more likely to be frowned upon and rejected by reviewers, even if they are scientifically correct. Moreover, negative result papers get scrutinized more than their positive result counterparts, reports Nature magazine.
In a study by Seth Leopold of the University of Washington, Seattle, two fake papers comparing relative benefits of two strategies of antibiotic treatment were composed. They were identical, but with one crucial difference: one said the methods were no different in efficiency, while the other claimed one was better than the other.
Both fake studies were of very high quality and conformed to the best standards of medical research. The two compared methods – a single dose of antibiotics before surgery versus a starter doze plus smaller doses over a 24-hour follow up period – are highly debated by medical scientists at the moment. Theoretically, both papers should have been of equal interest for reviewers, but that was not true.
The “one method better” paper received 98% positive reviews from reviewers of the American edition of Journal of Bone and Joint Surgery (JBJS). The “no difference” paper was recommended for publication by only 71%. Strikingly, these reviewers also gave the entirely identical methods section a full point advantage (on a scale of one to ten) in the positive paper. “There's no good explanation for that,” says Leopold.
Reviewers were also different in how well they found five intentional small mistakes planted in the papers. For the positive result paper, they caught only 0.3 errors per reviewer as compared to 0.7 errors per reviewer for the negative result paper.
The positive result bias is “a major problem for evidence-based medicine,” Leopold said while presenting his work at the Sixth International Congress of Peer Review and Biomedical Publication in Vancouver, British Columbia. It can skew journals towards good reviews of drugs, which may affect recommended treatments.
The study “just goes to show that peer review is done by biased, subjective people," says Liz Wager, managing director of the Sideview consultancy in Princes Risborough, UK, and chair of the UK-based Committee on Publication Ethics. “Everyone wants the new stuff to work – they want to believe.”
discuss it Show comments (1)
