Fake blood is seen in test tubes labelled ‘COVID-19’ in this photograph taken in March 2020. Photo: Reuters/Dado Ruvic/Illustration/File Photo.
Bengaluru: Before the COVID-19 epidemic took off in India, few people were interested in monitoring the goings-on at the Clinical Trials Registry – India (CTRI).
“There was hardly anyone looking at it except a few academics like us,” said Amar Jesani, editor of the Indian Journal of Medical Ethics (IJME) and a founder of the Forum for Medical Ethics Society.
But when pharmaceutical and Ayurveda companies began floating ‘cures’ for the viral disease based on fishy clinical trials they’d conducted, the Indian registry started to come under more scrutiny.
In June this year, yoga guru Baba Ramdev announced that his company Patanjali Ayurved had launched a drug called Coronil that he said could cure COVID-19. The supposedly Ayurvedic formulation had undergone a “randomised placebo controlled double blind clinical trial,” he said in a press conference, and that the trial had been registered with CTRI.
People who were curious about Ramdev’s claims looked up the trial on CTRI, and then began raising questions about the study’s conduct. After journalists raised a stink, the Ministry of AYUSH, which regulates the certification and sale of Ayurvedic products in India, pulled up Ramdev and prevented Coronil from being marketed as a cure for COVID-19.
Similarly, when Biocon Limited announced in July that the drug itolizumab had been found effective in treating patients with severe COVID-19, the trial’s information on CTRI became a tool to critique the reliability of the study’s results.
Indeed, CTRI has now become one of the go-to sources for people to track pandemic-related trials, of which there has been a surge since April. Clinical trials are research studies that test whether a new treatment, say a drug, vaccine or a new surgical method, can be administered to patients.
Where one COVID-19-related clinical trial was registered with CTRI between January to March 2020, there were 522 trials registered from April to August, according to data shared by Vishnu Vardhana Rao, the director of the Indian Council of Medical Research’s (ICMR’s) National Institute of Medical Statistics (NIMS), which hosts CTRI. Overall, CTRI has more than 28,000 registered trials.
CTRI has become a go-to source for people to track pandemic-related trials, of which there has been a surge since April.
The registry has a simple purpose. It is a free, online platform where anyone can view the records of clinical trials happening in the country. When CTRI was launched in 2007, registration was voluntary for two years. From 2009, the Drugs Controller General of India (DCGI) of the Central Drugs Standard Control Organisation (CDSCO) mandated all regulatory clinical trials being conducted in India to be registered with CTRI.
Once registered, some details of the planned study and the basic study design become public. It’s somewhat like a number entering a telephone directory, joining a list of names, others numbers and addresses of all businesses in a neighbourhood, so you know what’s out there.
“All registration says is what a trial intends to do and how people can get in touch with the researchers,” Nicholas DeVito, a researcher who studies trial registries at the University of Oxford’s Evidence-Based Medicine DataLab, told The Wire Science. “It tells you what research is being done in a country. That way, people can find trials to join and scientists know what other people are doing.”
The Indian registry set four objectives in its vision statement: to improve transparency and accountability; to improve the scientific validity of trials; to conform to accepted ethical standards; and to report all relevant results of registered trials.
However, experts say it isn’t achieving these goals.
“In the beginning, nobody expects perfection,” said Jesani. “What we expect is that in the course of time, it will try to achieve the objectives that are set. But there’s been disappointment there.”
§
I. Transparency and accountability
The Indian registry was created in response to growing concerns worldwide that crucial information about most clinical trials wasn’t available in the public domain – despite large numbers of trials being conducted in the country. Experts bemoaned trial investigators’ tendency to declare only positive or favourable results, or change the outcomes being reported, so the trial would seem more successful than it really was.
“Because there was no mandate to report details of trials, the drug companies were not going to report untoward events either,” said Prathap Tharyan, an adjunct professor in the clinical epidemiology unit of Christian Medical College, Vellore, and a member of the CTRI technical working group.
Registries like CTRI are meant to be an antidote to such selective reporting. Since April 2018, all trials have had to be prospectively registered – that is, before they start recruiting their first participant. This way, people can see what a trial plans to do before it starts to do that. And the data items that a trial registrant must disclose to the public plays a big part in this.
Registries like CTRI are meant to be an antidote to selective reporting.
CTRI has 21 mandatory fields, in addition to several voluntary ones, that largely comply with the WHO’s International Standards for Clinical Trial Registries. Compliance is essential because CTRI is a primary registry of the WHO Registry Network. CTRI’s compulsory fields include information like the public and scientific titles of the proposed study, details of contact persons, trial sponsors, type of study design, intervention being tested, outcomes being studied, places where the trial is planned, date when recruitment will start and the criteria for a participant’s inclusion and exclusion.
In addition, CTRI requires registrants to submit approval letters from the DCGI, and the ethics committees of each trial site. The staff also emails trial investigators and approves registration only after receiving responses from them. CTRI staff use these steps to verify if the trial is planned for real, and if it has received the requisite approvals to begin recruiting patients.
§
A database is, however, only as good as the data entered in it. “Each field in CTRI tells its own story, which can be part of a larger analysis of many trials, and therefore each one needs to be correctly filled,” said Gayatri Saberwal, a scientist with the Institute of Bioinformatics and Applied Biotechnology, Bengaluru.
However, CTRI’s records often have missing or incomplete information, Saberwal’s team found in a study published in 2019. For example, names of the principal investigator are sometimes missing or spelt in different ways. There are also variations in the names of the primary sponsor or other organisations involved in the trial.
Such bad records don’t necessarily mean that the trial is being done badly, or unethically so, Saberwal said – but complete and accurate CTRI records help create public trust in the clinical research ecosystem.
For example, if you wanted to know how many trials the American pharmaceutical company Bristol Myers Squibb has sponsored in India, you’d have to try out various versions of the name, including ‘Bristol Myers Squibb India Pvt. Ltd.’, ‘Bristol-Myers Squibb’ and ‘BristolMyers Squibb India pvt Ltd.’.
Complete and accurate CTRI records help create public trust in the clinical research ecosystem.
Similarly, if you wanted to check if a certain doctor was overseeing too many trials at the same time, you might never get the right answer: the doctor’s name might be missing from some trials or may have been spelt in other ways (a common issue in India, where the same name written in an Indian language is often transliterated to different spellings in English).
According to Saberwal, we can avoid these errors by pre-registering individuals’ and organisations’ names, so they can then be selected from drop-down menus instead of being free-form text.
This said, her study also found the fraction of studies with poor CTRI data has gone down in recent years. Rao attributed this improvement to both greater awareness among researchers on trial registration and more experience among CTRI staff to review trial records. “The software was also upgraded and made more user-friendly based on experiences shared by registrants,” Rao said.
Also read: There Are More Questions We Need to Ask About the COVAXIN Clinical Trials
§
But CTRI isn’t exactly user-friendly for people simply viewing the registry. For one, the search feature doesn’t return reliable results. “If it didn’t give me a result, I’d know that I don’t have the answer,” Saberwal said. “But if it gives me a wrong answer, then that’s misleading.”
The registry also doesn’t allow people to download all the records at one go for analysis. “You can search trials only by adding keywords,” said Jaykaran Charan, an associate professor at the department of pharmacology in AIIMS, Jodhpur. “For COVID-19, it was easy, but if I want to see all the trials on the platform – that is not possible.”
Thanks in part to these problems, Clinical Trials Watch, a periodic analysis of clinical trials in India that Jesani and his colleagues had been publishing in IJME from 2009, had to be called off. “We wanted to find out if we were doing research” for diseases of priority “in India,” Jesani said. “But organising CTRI data and cleaning it took so much time. That was when we realised that this database is not telling us much.”
There is also no guarantee that a trial registered with CTRI presents the latest information. This is despite WHO requiring primary registries to regularly perform quality-control audits to assess how complete and accurate the registry’s records are.
Biocon’s trial, for example, was registered on May 1, according to the CTRI entry. The company held a press conference on July 13 to announce the findings of its trial. On August 18, several changes appeared in the CTRI entry: six trial sites became four; the principal investigator changed; and the outcomes that the trial intended to study shot up from one primary outcome to six.
Biocon’s chief medical officer Sandeep Athalye told The Wire Science that the team obtained approvals for the amendments from both the DCGI and the ethics committees of each site before starting the trial, as is required. However, the request to update the CTRI entry was made more recently and took about three weeks to reflect, he said.
Once a trial is registered, CTRI locks most data fields. Modifications are only allowed after the registrant requests a field to be opened together with good reasons. This back and forth takes time, Athalye said.
Moreover, the CTRI staff themselves don’t actively follow up with the sponsor to ensure the latest version is up there on the registry, he said. “We ensured that the latest thing is uploaded, but if a sponsor forgets to update an amendment, the first version will remain there forever.”
There is no guarantee that a trial registered with CTRI presents the latest information.
Without a timely update of the registered trial, it’s hard to know if the details reflect the trial’s actual conduct.
CTRI does provide an audit trail of the changes made by maintaining a record of when different modifications happened. However, exact details about how a field was modified aren’t easily deciphered.
By contrast, the US’s clinicaltrials.gov (CTG), one of the biggest registries in the world, allows people to see each version of the trial and compare them side by side. “You can see how things changed with it very clearly,” said DeVito. “But the fact that CTRI shows that changes were made at all is a positive. The EU Clinical Trials Register, for instance, has no version control.”
§
Then there is the question of whether all clinical trials that are required by law to register with CTRI do so. A recent study by Saberwal’s team compared CTG and CTRI, and found that thousands of CTG records that listed India as one of the countries of recruitment did not have a corresponding CTRI ID, contravening best practice.
When the team tried to find a corresponding CTRI record for the CTG-registered trials using fields like the study’s title, their algorithms predicted that between 50 and 300 of the 581 CTG trials they scrutinised weren’t registered with CTRI. Ergo, some regulatory trials may have broken the law.
“Our modeling exercise tells us that this phenomenon of unregistered trials exists, but it cannot identify those trials,” Saberwal said. “For that the DCGI has to launch its own inquiry if it is serious about it.”
There has to be a mechanism to reconcile the register of the CDSCO with CTRI, said Jesani. “If there were certain numbers of trials that were not registered on CTRI, but those trials were given permission by CDSCO, that should be easy to find out.”
Both V.G. Somani, the DCGI, and S. Eswara Reddy, the joint Drugs Controller (India), did not respond to emailed questions from The Wire Science. As for CTRI itself – the registry is a facilitator of trial registration, Rao said, but can’t – and doesn’t – enforce it.
§
II. The scientific validity of trials
A clinical trial uses scientific methods to judge how effective and safe a drug or a treatment is in people. So details of the trial, including the research method it proposes to employ, its sample size, the profile of the participants that will be included or excluded, are critical to determine the trial’s validity. That is, to see if the inference drawn from the trial is valid based on how the study was conducted.
Consider, for example, a randomised controlled trial (RCT), considered to be the ‘gold standard’ for clinical trials – the same design both Patanjali and Biocon claim to have employed according to their respective CTRI entries.
In an RCT, investigators randomly allocate participants to either the group that receives the intervention or to a control group that doesn’t (which instead receives a placebo or standard care), but is nearly identical in all other ways. The investigators then administer the treatment being studied and observe the two groups over a period of time. At the end, they compare the groups in terms of outcomes – or endpoints – that they had set out to study with the trial. Any bias, say while deciding which group a participant should be in or not accounting for confounding factors that could explain the trial’s results, could threaten the trial’s validity.
CTRI does have fields that provide insight into some of these aspects. But not all fields are mandatory – including the method of randomisation; whether the patients knew which group they were getting into or whether this information was blinded or masked; and the method of concealment, if there was blinding. If these fields are blank, it’s hard to say how useful an RCT’s results are.
Biocon left the method of random sequence generation field empty when it first registered its trial. This would have been a crucial bit of information, especially since their sample size of 30 patients was very small. As science journalist Priyanka Pulla subsequently found for The Wire Science, five patients hadn’t been randomised but the CTRI entry didn’t mention this.
Some regulatory trials may have broken the law.
These discrepancies matter. Many international and national medical journals require a CTRI registration ID for a study to be published. In theory, comparing the trial method listed in the paper with whatever was listed on CTRI could help editors verify if the authors have reported all outcomes correctly. But that’s difficult to do if you can’t be sure if the data on CTRI is accurate – although the mismatches themselves, and the change history, hint at problems that the reviewers can question.
CTRI also doesn’t shed light on other useful aspects of the study’s protocol. For a phase 3 trial, for instance, you don’t get information about the outcomes of phase 1 or phase 2 trials for the drug – helpful to determine the drug’s safety in patients – according to Jesani.
However, some experts said it isn’t the registry’s job to ensure the trial’s methods are valid. “At one point, one of the delays in getting the trials registered was because the staff were actually looking at whether the scientific content was correct or not,” said Tharyan. “Then we decided that the job of the registry is to register what these guys are saying. Whether they are going to do false trials is the responsibility of the individual ethics committees, because they have approved the trial.”
DeVito agreed. “A trial registry is an accounting of what’s going on in the country. If you want to be judge and juror on whether a trial should happen, that’s the ethics committee’s job.”
§
III. Accepted ethical standards
In a clinical trial, science and ethics are intertwined. “You can’t have an ethically run trial that is scientifically rubbish, and vice versa,” said Tharyan.
If a study is badly designed, you will know you won’t be able to trust its results. So should the trial be allowed to recruit patients in the first place? Is there a conflict of interest – say, if an ethics committee member has close ties with the sponsor? Does the patient really understand the risks of participating in a trial?
For biomedical health research in India, ICMR has developed ethical guidelines that clinical researchers and ethics committees are advised to follow. ICMR has also drafted a separate set of guidelines for ethics committees reviewing health research during the pandemic, to ensure that research isn’t fast-track at the cost of “scientific validity and ethical requirements”.
And CTRI facilitates ICMR’s guidelines in two ways: it publicly lists the names of all ethics committees involved in the trial, and discloses each approval’s status. The approval letters themselves aren’t published. If a trial is being conducted in multiple places, an approval letter from just one site suffices for the trial to be registered.
“You can’t have an ethically run trial that is scientifically rubbish, and vice versa.”
This limited information doesn’t reveal much, Jesani said. “You don’t come to know who the ethics committee members were when this protocol was reviewed and whether any member had any kind of conflict of interest.”
For multicenter trials, registrants can remove a site from their CTRI entry without the reasons showing up on CTRI. The ethics committee for that site could have, for example, objected to the protocol or asked for amendments – but there’s no way to know at the moment.
“Suppose a clinical trial is taking place in five centres and one ethics committee has raised serious issues – the other ethics committees never come to know about it,” said Jesani. “If each committee’s position is reflected on CTRI, then other ethics committees can see it. There has to be a system where the committees can talk to each other.”
§
IV. Reporting results
One of the 24 WHO-specified data-items that must exist in a primary registry is ‘summary results’. This includes details like the data collected for all participants at the trial’s start, adverse health effects due to the treatment and final outcomes. And according to the WHO, a trial’s summary results must be posted in a primary registry within 12 months of completing the study.
In 2017, ICMR signed a joint statement with WHO agreeing to this requirement. Three years later, there is still no field to submit trial results.
NIMS’s director Rao defended its absence saying that it was under development, and was undergoing beta-testing when the “pandemic altered priorities”. “Software had to be updated to enable processing of COVID-19 studies separately. … Result disclosure will be implemented shortly.”
In its current form, CTRI may not be the best registry out there – but it’s not the worst either.
In the meantime, the absence of a summary of results is a problem. “Say I run 10 trials with a given drug for a given indication,” Saberwal said. “Only two are positive, and I publish only those. The world thinks that this is a great drug, but I hid eight of the trials that had negative results.”
The Cochrane collaboration, a global network of researchers and doctors, for example, regularly and systematically reviews RCTs. Clinicians then use these reviews to understand how safe and effective a treatment could be. The availability of both positive and negative results is key for such work.
It “gives us the scope to say that before you do further research, there are at least 10 trials going on addressing this question — let’s wait for those results to be out before you fund further research,” said Tharyan, until recently director of Cochrane South Asia.
Also read: How a Controversial Trial to Reverse Brain Death Slipped Through Regulatory Cracks
§
CTRI’s future
CTRI could become a powerful tool to answer public health questions. It could, for example, be used to make sense of India’s research priorities and how they may need to be changed.
But for the ‘could become’ to be ‘is’, Jesani said, CTRI data needs to be complete, up to date and transparent.
Charan, of AIIMS Jodhpur, recently used CTRI to check the sort of trials being conducted to treat or prevent COVID-19. As on July 11, he found more than 61% of the trials were AYUSH-related. “This is a welcome finding because we are mainstreaming AYUSH in evidence generation,” Charan said. But to judge the quality of a trial, he said he’d need to see the results.
In its current form, CTRI may not be the best registry out there – but it’s not the worst either. Saberwal’s team also compared CTRI with 17 other public registries, checked for user-friendliness and how closely they followed WHO guidelines.
“As part of this project, we have scored each registry on various parameters, and in terms of its total score, India is unfortunately roughly in the middle,” she said. “One would have wished it to be much higher.”
This could happen if India fixes CTRI’s shortcomings – even the known ones. And the benefits wouldn’t accrue only to pharmaceutical companies or researchers. Chirag Trivedi, president of the Indian Society for Clinical Research, said people could also become more aware of trials as a result. “Just recently, we received a request from a person who wanted to know how he could participate in a COVID-19 vaccine study. We referred the person to the CTRI website.”
Good CTRI records also help with the science. Take, for example, the ability to scrutinise all COVID-19 trials because they exist on CTRI. “But we should actually be able to follow up and see how many of these trials get published, what their results are and if we can trust these results,” Tharyan said. “The trial registration process is just one of the templates by which science can improve.”
This reportage was supported by a grant from the Thakur Family Foundation. The foundation has not exercised any editorial control over the contents of this report.
Shreya Dasgupta (@ShreyaDasgupta) is an independent science writer based in Bengaluru, India. Her work has appeared in Mongabay, Nature, BBC Earth, Smithsonian.com, New Scientist, Ensia, and other publications.