Publish to flourish

Explosion of knowledge or just junk science?

In 2006 alone, approximately 1.3 million peer-reviewed scientific articles were published, aided by a large rise in the number of available scientific journals from 16 000 in 2001 to 23 750 by 2006.³ This avalanche of papers raises a question. Are we witnessing an explosion of scientific knowledge or just the accumulation of wasteful publications and junk science? The value of a published paper is commonly measured by its citation index. However, data show that only 45% of the articles published in the 4500 top scientific journals are cited within the first five years of publication, a figure that appears to be dropping steadily.⁴ Only 42% receive more than one citation. Add to this the woe of self-citation, which accounts for between 5% and 25% of all citations,⁵ and it appears that most published papers are inconsequential to science and simply pad the curricula vitae of researchers.

The increased number of published papers may in some cases be related to dubious research practices such as salami slicing, whereby basic research is split into many fragments that allow individual publication. This is the concept of the so-called least publishable unit. Researchers may at times also publish the same material in different journals with different key words, captions and co-author variation on each occasion, thus making detection difficult by database scanners. Plagiarism, for example, is so rampant, that it appears many authors follow the quote attributed to Wilson Mizner, “If you steal from one author, it’s plagiarism; if you steal from many, it’s research.”⁶ Plagiarism and autoplagiarism, dual publication by the same author in the same or different language, are becoming so common that special programmes to identify the techniques are increasingly used by editors.^7,8

The increasing numbers game has led to the procedure of evaluating an article by a citation index as well as an H-index for the author. This, in turn, has led to citation fever and the development of many innovative techniques to improve the citation index of an article. The most common is self-citation where the first paragraphs of an Introduction and Discussion are dedicated to the citation of the author’s previous articles. Scientists can also form a mutual-citation clubwhere they can improve their own and their friends’ citation index, by mutually citing each other’s work. There is also the opposite problem where researchers neglect to quote important work by their competitors in order to reduce their opponent’s citation index; this is selective citation amnesia. Many indices are open to manipulation, using techniques that are widely practised.

Research and publishing misconduct

The frequency with which questionable practices are detected and reported is becoming more common and potentially jeopardising the sanctity of research. Such behaviour can form a spectrum of activities such as simple carelessness, bias, adjusting data to improve results, selective reporting, falsification of data and outright fraud. Although in many cases it is difficult to differentiate between scientific carelessness and wilful fraud, the intention to purposefully deceive is the key difference. Going by this definition, it appears that fraudulent scientific behaviour is more common than was once thought. In a systematic review and meta-analysis of survey data into how many scientists fabricate and falsify research, it was alarming to note that while only 1.97% of scientists admitted to having fabricated, falsified or modified data or results at least once, a serious form of misconduct by any standards, up to 33.7% admitted to other questionable research practices.⁹ In a sample of post-doctoral fellows at the University of California, San Francisco, 3.4% admitted they had modified data in the past and 17% were willing to select or omit data to improve their results.¹⁰ Meanwhile 81% were willing to select, omit or fabricate data to win a grant or publish a paper.¹¹ Three surveys documented scientists admitting to modifying or altering research data in order to make their research more impressive. Although many researchers did not consider data improvement as falsification, the hazy boundary between right and wrong, while biasing results towards a desired outcome, is increasingly crossed by many researchers.^12-14 As most of the surveys were based on anonymous self-reporting, the actual frequency of research misconduct may be even higher. Human behaviour shows that those responsible for acts of misconduct are likely to commit them more than once. This perhaps implies that the rate of misconduct may be very high among medical practitioners and researchers.

The problem is likely to be more common in the area of so-called grey science, scientific material presented from the podium, or presented in posters at prestigious conferences. These are now published as abstracts in many journals and thus indirectly obtain the sanctity of peer-reviewed publication. In financially troubled times, many conferences aim to achieve higher attendance by accepting more papers, at least at the poster level. Consequently, the peer-review process may not be as rigorous as it once was. The high level of questionable research practices possible in papers presented at meetings is of obvious concern.

The published paper is no longer sacred

The peer-review process, while struggling to maintain quality, frequently fails to maintain scientific integrity. Honesty, good faith and good research practices, the cornerstones of science, cannot be policed or verified by reviewers. The increasing number of retractions witnessed in recent years shows evidence of this.

Retraction Watch,¹⁵ a blog that tracks scientific retractions, has documented more than 250 retractions over a 16-month period, prompting them to declare 2011 the ‘Year of Retraction’. Retractions have increased 15-fold in the past decade.¹⁶ While many retractions may be due to poor research practices such as failing to obtain ethical committee approval, some may be the result of falsification and fabrication of data. A recent and extreme example, which has shaken the scientific world, is that of the South Korean stem-cell researcher Woo-Suk Hwang. The case stands as a perfect example of research misconduct and highlights the woes of modern-day scientific publishing. Hwang et al published two highly cited papers in Science in 2004 and 2005,^17,18 which reported the concept of therapeutic cloning in humans. In the first paper, the researchers claimed isolation of the first human embryonic stem cell line derived from somatic cell nuclear transfer and in the second they reported refinement of the process, which made clinical application possible. Hwang soon acquired the status of a national hero. However, it rapidly became obvious that not only had unethical practices been adopted in obtaining egg donors but DNA finger-printing of the cell lines showed the entire scientific content to be fabricated and untrue. Although Science retracted the articles, these events exposed the fact that researchers can fabricate data with ease and achieve publication in respected journals.

This case highlights additional, important ethical considerations. Hwang was forced to resign from Seoul National University in December 2005 and was officially dismissed in March 2006. Nevertheless, he remained active in the research field and successfully submitted three manuscripts on behalf of his university in the same year. These were subsequently published. Currently he has resumed research in a private facility in South Korea and further papers have appeared under his name.

This raises an interesting ethical issue of whether a researcher with a proven record of scientific fraud can be trusted to publish in areas of high importance. It is perhaps time for the scientific community to decide if authors should be prevented from submitting manuscripts to scientific journals once clear fraud and fabrication have been established.

Peer review needs review

There is a strong suggestion that the current review procedures into fraud and data manipulation, and the efficacy of the peer-review process, need evaluation. The peer-review process is thought to have started when the Royal Society of London published Philosophical Transactions in 1665.^19,20 Historically, the process is considered to serve as the goalkeeper of scientific quality but not of scientific integrity. If the review process is the gateway for quality, then it is important to have high quality reviewers. The entire process is under strain as there are too many journals seeking too few reviewers who have the time, expertise and willingness to provide quality reviews. Most reviewers have no training in the fundamentals of good reviewing while real experts in any field are also overburdened with many other duties such as teaching, mentoring students, their own research, and administration, to review regularly. They are often forced to ask their junior colleagues, sometimes even their PhD students, to help. Studies have shown that many reviewers of even important journals are unable to distinguish between what is good and what is excellent.²¹ A review of the editorial processes for a number of journals – Cardiovascular Research,²²Radiology,²³ and the Journal of Clinical Investigation²⁴ – has exposed poor agreement between different reviewers of the same manuscript. In practice, it is frequently found that what is assessed as excellent by one reviewer may be considered unworthy of publication by another. Submitted manuscripts that are rejected by one journal will often be published elsewhere, thereby obtaining the status of a peer-reviewed scholarly publication.

Is there a solution?

For this process to change, the system of rewards must change. Applicants to important posts must be assessed on the basis of their best four or five papers rather than enumerating all their publications. This might dissuade young researchers from joining the treadmill of the numbers game and encourage focused publishing of high-quality research instead. A single article with a high citation index in a high impact factor journal should be of greater value than five or six articles in less frequently cited journals. The length of the papers might also be reduced, as is seen with Nature and Science. Meanwhile libraries should be encouraged to cease their subscriptions to journals that do not register a good impact. The process of review and method of publication of journals must undergo scrutiny and necessary change. In addition, methods to detect fraud and identify questionable research practices must be strengthened and perhaps be separated from the review process.

Of course, a lasting solution might come from reforms within. Young investigators must be taught to appreciate good research morals as well as sound methodologies. Professors must teach by example and build a research environment that will foster good research practices. Presently, most researchers look at paper publishing not as a means to share and disseminate their work but as a method for professional advancement or as a means to obtain a grant. It is time that the research community recognised that the few cases that have been exposed represent the tip of an iceberg. The scientific community must look at a means of internal cleansing. Otherwise the very foundations of evidence-based medicine will be destroyed.