Is every scientific study genuine? BMJ throws light on ‘data fraud’ by countries, including India

In an article in BMJ, former editor Richard Smith says prevalence of fraudulent studies has reached a point where one can now assume health research is fraudulent until proven otherwise.

Representational image | Biological samples are placed in a nitrogen freeze chamber in a cryogenic cold storage lab | Nathan Laine/Bloomberg

New Delhi: In a year that saw an explosion in clinical trials and scientific data due to the Covid-19 pandemic, the question of data accuracy is growing to be a troubling matter.

According to the BMJ medical journal, as much as 54 per cent of clinical trials published by Indian researchers over three years in a single journal were found to have fraudulent data. The trend was similar in other countries — 100 per cent in Egypt, 75 per cent in Iran, 46 per cent in China, 40 per cent in Turkey, 25 per cent in South Korea, and 18 per cent in Japan — raising concerns about the integrity of research being done here.

Last year, when scientific studies drew much more global attention than usual, The Lancet and New England Journal retracted two studies — one on hydroxychloroquine and another on the effect of some cardiovascular drugs in Covid-19 treatment — after they were found to be based on fraudulent data.

Richard Smith, former editor of the BMJ, noted that data fraud has become so prevalent that “the time may have come to stop assuming that research actually happened and is honestly reported, and assume that the research is fraudulent until there is some evidence to support it having happened and been honestly reported”.

In an article dated 5 July, Smith cited a study by John Carlisle, a UK-based consultant anaesthetist who analysed 526 trials submitted to the journal Anaesthesia between February 2017 and March 2020, found that 73 (14 per cent) had false data, and 43 (8 per cent) were ‘zombie’, i.e., research that was completely flawed.

When Carlisle examined individual patient data in 153 of these studies, 44 per cent had untrustworthy data and 26 per cent were zombie trials. Many of these trials came from Egypt, China, India, Iran, Japan, South Korea, and Turkey.

Later, when John Ioannidis, a professor at Stanford University, examined individual patient data from trials submitted from these countries to Anaesthesia, 54 per cent of the studies from India were found to be flawed.

Experts in India, however, say using a broad brush to paint all scientific research as problematic or potentially fraudulent is doing a disservice to the research sector. In some cases, it might just be a question of how one defines data as flawed. Most, though, agree that the gatekeeping process needs to be more robust and exacting.

The problem in India 

Soumyadeep Bhaumik, co-head of the meta-research and evidence synthesis unit at George Institute for Global Health, New Delhi, told ThePrint that although data manipulation is a problem in India, it is unfair to quantify its extent based on the data mentioned in the BMJ piece.

“A key challenge of this type of analysis is how one defines that data is flawed. There may be a lot of reasons for a data set to not match the criteria set by such studies,” Bhaumik said.

The narrative of treating all research from certain countries as ‘fraudulent by default’ will do a disservice to the field of research, he added. “It has a colonial lens. Researchers in other countries might actually be doing better data fraud and avoiding detection.”

Speaking to ThePrint, K. Srinath Reddy, former head of the Department of Cardiology at All India Institute of Medical Sciences (AIIMS), Delhi, noted that in India, by linking promotions to the body of research one produces, it is incentivising the practice of poorly conducted studies.

“Perverse incentives for publishing poorly conducted or even falsified research may increase because the Medical Council of India, and now also National Medical Commission, link the number of research papers to promotions in medical colleges,” he said.

Bhaumik pointed out that gatekeepers in scientific research often do not work, and the process as a whole needs overhauling. The first gatekeeper is the institutional ethics committee that approves a study.

“Unfortunately these ethics committees are seen as just another bureaucratic hurdle for most scientists. Many ethics committees do not check whether all clinical trial related protocols are being followed,” he said.

The second gatekeeper is the journal, and the reviewers who go through the merits of the study before it is published.

“Editors and reviewers do this work for free. They are not paid to take out time to rigorously scrutinise the data,” Bhaumik said.

Samrat Ray, a consultant at the Department of Surgical Gastroenterology and Liver Transplantation, Sir Ganga Ram Hospital, said: “Most medical centres in India lack a definitive systematic inventory of medical records and data that could be easily retrieved by researchers for their studies. However, in medical colleges, promotion to a higher grade requires the faculty to produce a particular number of papers.”

Therefore, he added, “lack of a systematic honest database along with the pressure of producing papers could be a possible driving force to manipulate data and produce any kind of research (with no assurance of the quality and conclusions)”.

Tackling the problem

Instead of relying on one or two popular studies, public policy decisions should be based on evidence synthesis — a method which pools results from independent studies.

“Respect for research methodology should be inculcated right from the formative years in the undergraduate stage. Scientific temperament should be built, so that the mindset to question results, including one’s own, is developed to a high level of objectivity,” Reddy said.

“People often view research just as a ladder for promotion. Instead of quantitative measures of research, impact on policy and practice has to be assessed.”

He also said the most premiere institutes of the country do not include a structured training of research methodology to the undergraduates or postgraduates in their curriculum. In the two- and three- tier cities, post-graduate theses are mostly repetition of the previously published theses with some modifications done in the title.

“In fact, this practice has led to mushrooming of many professional ‘thesis makers’, who make money out of writing up an entire thesis for the post graduate student using completely fabricated data,” Ray said, adding that there are no quality check-points on such fraudulent practices.

Not a new occurrence

Instances of data fraud are neither new nor rare. In 1992, Dr Ram B. Singh, a private practitioner from Moradabad, co-authored an article in the BMJ, which concluded that a year of maintaining a low fat, fibre-rich diet almost halved the risk of death from all causes among people who were susceptible to heart attacks.

However, soon after its publication, the BMJ received several letters questioning the integrity of the data. Singh became the centre of an international investigation spanning well over a decade.

Although the investigation could not conclusively prove misconduct, it did show that several of Singh’s subsequently published work also included questionable data.

Under Smith, the BMJ published the details of this decade-long investigation in order to show that when researchers came from outside the jurisdiction of official research bodies, journal editors could do little to address scientific misconduct.

(Edited by Manasa Mohan)