Can you believe all that you read in the medical journals?

Trauma surgeons are familiar with the use of mannitol in treating head injury. What doses they use are based on a combination of experience, knowledge gleaned from the scientific literature and information from guidelines. Those preferring a high-dose regime may have been influenced by a Cochrane review, published in 2005.¹ This commended the use of a high dose, influenced greatly by three studies from the same lead author published in high impact neurosurgical journals.^2-4 In 2007, serious doubts were cast on the veracity of these studies. It was alleged the data had been fabricated but the lead author is dead, the co-authors claim never to have had first-hand knowledge of the patients involved and there is no employing institution extant to mount an investigation.⁵ High dosages of mannitol may be of no additional value compared with standard dose or may actually be harmful.⁶ Thus, dishonesty by a researcher, non-adherence by co-authors to recognise authorship criteria, and failure of reviewers and editors to detect fraud can result in the publication of data that are likely fraudulent. One result was that the guidelines developed by Wakal, Roberts and Scierhout¹ for the Cochrane Collaboration unwittingly laundered dirty data into clean clinical recommendations. Head trauma is serious business. If guidelines were wrong, because the supporting data were fraudulent, how many patients were harmed by inappropriate treatment?

By and large, physicians and surgeons trust what they read, even if they take authors’ conclusions with a pinch of salt. There is a world of difference, however, between being cautious about the implications of what you read and being defrauded by dishonest researchers.

In 2008, an investigation by Baystate Medical Center (Springfield, Massachusetts, USA) of its former head of anaesthesia, Scott Reuben, found that he had invented all or part of the data in a large series of studies since 1996, with the Journal of Bone and Joint Surgery[Am]⁷ and Anesthesia and Analgesia⁸ subsequently posting retractions. There were 21 papers retracted for fabricated data. Reuben’s research purportedly demonstrated excellent post-operative pain relief from the combination of a COX-2 inhibitor with pregabalin or gabapentin.⁹ His work in multimodal analgesia was welcomed by orthopaedic surgeons.¹⁰The findings are now in doubt, pending replication of his research.

The current record for retractions is probably held by Joachim Boldt, ex-professor of anaesthesiology in Rheinland-Pfalz (Germany), 89 of whose publications were retracted in 2011 after his institution found that none of the studies had IRB (ethics committee) approval.⁹ As investigations continued into the veracity of the data, the Association of Surgeons, the Intensive Care Society and numerous other professional bodies felt constrained to withdraw and amend their published guidelines on intravenous fluid therapy.

Such blatant fraud may not be common and has a habit of being detected, eventually, but any such ‘self-correction’ does not mean that patients have been unharmed. A survey of all 788 English language papers involving human research retracted from PubMed between 2000 and 2010 showed that 28 000 subjects had been recruited, the papers cited over 5000 times and the studies provoking or cited by 851 secondary studies which had enrolled 400 000 patients.¹¹

Along with fabrication and falsification, journal editors generally include plagiarism as serious misconduct. There are certainly a lot of plagiarists about – even when one excludes those who claim inadvertent cutting and pasting, perhaps because of unfamiliarity with the language of the journal concerned. In the last two years alone, papers by Perugian orthopaedic surgeon, Bernardino Saccomanni on vertebroplasty, elbow arthroplasty and osteoarthritis of the knee have been retracted from Osteoporosis International, Clinical Rheumatology and Musculoskeletal Surgery for blatant plagiarism, and intrepid followers of the blog Retraction Watch¹² have reported many more examples of unretracted plagiarised papers, using the simple expedient of typing a sentence or two into Google and watching the original provenance appear.

Much of the debate about research and publication misconduct has been initiated by journal editors. One result has been the establishment of COPE, the Committee on Publication Ethics, representing some 7000 editors, which publishes guidance on how to deal with suspected misconduct.¹³ Some experienced and senior editors express major concerns about probity. Marcia Angell, a former editor of the New England Journal of Medicine, stated in a book review about pharmaceutical company marketing: “It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion which I reached slowly and reluctantly over my two decades as an editor of NEJM.”¹⁴

Richard Smith, former editor of the British Medical Journal included in the summary of his book, The Trouble with Medical Journals, the following comment: “It’s increasingly apparent that many of the studies journals contain are fraudulent, and yet the scientific community has not responded adequately to the problem of fraud.”¹⁵

I am not surprised that editors agonise over misconduct; my own experience as an editor taught me that handling such allegations could take up as much time as processing many hundreds of blameless submissions. Unfortunately, communications with the authors and their employing institutions often lead to a frustrating impasse, as demonstrated by the experience when assessing the veracity of studies supporting high-dose mannitol for head trauma.

Many scientists and clinicians cling to a ‘bad apple’ concept of research misconduct. However, even if we accept that megafraudsters are few and far between, the question arises just how common is research misconduct in general? Fanelli¹⁶ conducted a systematic review of surveys asking scientists about their own and their colleagues’ behaviour. A total of 1.97% admitted falsifying, fabricating or otherwise improperly manipulating data. Meanwhile, 14% reported they were personally aware of such behaviour by others. When asked about ‘questionable research practices,’ which include selective reporting, incomplete data analysis, modification of images, failure to obtain ethical approval, not declaring a significant financial conflict of interest, using ghost authors, etc., 33.7% admitted such actions and ascribed them to 72% of colleagues.¹⁶

Extrapolating from their trawl of MedLine citations, Errami et al¹⁷ suggest that as many as 117 500 are duplicate publications. This figure needs to be confirmed by further analysis but whatever the frequency, duplication is not just an underhand but harmless way of enhancing a curriculum vitae. It fundamentally corrupts the scientific record. For example, in a systematic review of studies on the efficacy of ondansetron in gastro-oesophageal reflux, the effect of including duplicated studies reduced the number needed to treat (NNT) from 9.5 to 4.9. Curiously, the duplicated publications were more favourable to the drug than those published only once.¹⁸

When inconvenient data are excluded, vital clinical information can be hidden. For example, the grossly excess mortality of elderly demented patients treated with rofecoxib was discovered only after original data were uncovered following a US Freedom of Information Act request. Comparing the final draft of the study with the published version showed that a number of apparent guest authors had been added to the original list of authors, who were all employees of the pharmaceutical manufacturer involved. Additionally, review articles had been prepared by these ‘ghosts’ presumably as a basis for academics to submit them to journals under their own names. In responding to these matters, the editors of the Journal of the American Medical Association stated: “Public trust for clinical research is in great jeopardy.”¹⁹

Medical editors have proposed various strategies to clean up the literature. Trial registration, now a requirement of nearly all journals, has gone some way towards preventing the burial of bad news. Some editors have devised risk stratification checklists of submitted papers for extra scrutiny, for example where it seems unlikely that the authors could have been funded to the extent that seems necessary given the nature of the trial. While still relying on their honesty, requiring authors to state their precise contribution to a paper is one method of reducing ghost and guest authorship. The hope is that by introducing transparency, undeclared conflicts of interest might be diminished. Others have called for raw data to be made available to readers, or at least, securely stored in a case retrieval system. Many of the better resourced journals are now using plagiarism detection and image manipulation software as a routine.

Relatively straightforward statistical tests can be used to investigate suspicious data. Thus, when finding unusual clustering of coefficients of variation in the data presented in 84 papers by a group of biochemists, Hudes, McCann and Ames²⁰ commented that they could find: “no statistical or biological explanation.” A similar technique was used to expose fabricated data in 169 randomised trials by a disgraced professor of anaesthesiology.²¹

However, none of these is foolproof. The determined fraudster or proponent of questionable practices can usually find ways round these hazards, particularly if financial rewards, direct or indirect, are likely to follow publication.

In the end, clinicians need to have the same response to the medical literature as to the media generally. Not everything you read is true, whether by accident or intent. Scepticism is an important component of science, and a necessary clinical tool to protect your patients from the effects of research fraud.