Photo: Dan Asaki/Unsplash.
Bengaluru: As the world scrambled to find a cure for COVID-19, the pandemic gave us a real-time lesson on the importance of clinical trials. It is through these trials that we now know many experimental drugs and therapies, like plasma therapy, don’t work despite what scientists first believed. It is also through clinical trials that we now have COVID-19 vaccines out in the world.
But for these significant contributions, the actual conduct of clinical trials in India has largely stayed opaque. There have, however, been media reports of ethical and scientific violations in some trials, as well as questionable study designs.
For example, reports of ethical and protocol violations at People’s Hospital in Bhopal, which was one of the sites of Bharat Biotech’s phase 3 trial of its vaccine candidate Covaxin. In another controversy, Flores Hospital in Ghaziabad, one of the sites of the phase 3 trial of Zydus Cadila’s ZyCoV-D vaccine candidate, reportedly outsourced part of its trial to an unregistered lab, which then went on to illegally administer vaccine shots to local people in the guise of a free vaccine camp. In July 2020, when the Indian drug regulator accorded restricted emergency use approval to Biocon’s itolizumab, independent experts roundly criticised various discrepancies in the drug’s trial design and conduct.
If there’s one thing these cases have highlighted, it’s the importance of continuously monitoring a trial while it is still going on. In theory, there are two independent bodies that should be doing this: the sponsor-appointed but otherwise independent data and safety monitoring board (DSMB) [footnote]Although DSMBs are not appointed for every study[/footnote] and the institutional ethics committee that approved the trial in the first place.
The ethics committee has the bigger role of the two, especially vis-à-vis protecting the participants’ wellbeing and ensuring the study is conducted ethically.
The Indian Council of Medical Research, for example, notes that ethics committees
“… are entrusted with the responsibility to undertake the ethical review of research proposals prior to initiation, and also have a continuing responsibility to regularly monitor the approved research to ensure ethical compliance during the conduct of research and to ensure that the rights, safety and well-being of research participants is protected.”
Most ethics committees in India, however, monitor trials only passively. They review annual reports of the study that the principal investigator submits to them. They don’t actually go to the hospital or site where the trial is being conducted.
Imagine buying an under-construction house, and reviewing its progress through an annual report the contractor submits – but never visiting it to see if the construction and quality are in line with the original plan.
Does such off-site monitoring suffice to catch violations in clinical trials? No – according to the experience of a few ethics committees in the country that do perform on-site checks.
The institutional ethics committee at the King Edward Memorial (KEM) Hospital, Mumbai, is one of them. In two published studies (this and this), committee members detailed the kinds of mistakes they found when they visited trial sites.
The most common forms of violation involved the informed consent process – a central part of a clinical trial in which the investigator explains to a potential participant what the study is about, how it will be conducted, what the risks and potential benefits of participation could be, and what the participant’s rights are.
At some sites, for example, committee members found that the investigator used a version of the informed consent document that the committee hadn’t approved. At other sites, the documents missed or had incorrect signatures. Such flaws typically suggest that the research team doesn’t understand the purpose of the consent process.
Ethics committee members in fact found instances where the principal investigator – usually a doctor appointed by the sponsor to lead the study – was either unaware of the study protocol or didn’t know the guidelines by which she would have to obtain the consent of vulnerable people.
Sometimes, the documents containing the data pertaining to each trial participant weren’t stored properly or serious adverse events weren’t reported on time.
There were also instances in which investigators had deviated from the approved protocol, putting patients at risk. Every clinical trial has inclusion and exclusion criteria that determine which participants can or can’t be recruited. But the ethics committee found studies in which the study team either hadn’t followed these criteria or had recruited more participants than what had been approved for that site.
There’s no doubt that on-site monitoring catches more errors than passive monitoring – yet Indian regulations don’t mandate it. This said, it is compulsory for all ethics committees overseeing regulatory trials to register with the Central Drug Standards Control Organisation (CDSCO).
To improve the quality of their functioning, some committees, like KEM’s, have voluntarily got themselves accredited from international agencies like the Strategic Initiative for Developing Capacity in Ethical Review (SIDCER) and the National Accreditation Board for Hospitals & Healthcare Providers (NABH) in India.
(More than 1,200 ethics committees are registered with the CDSCO, but only around 170 are NABH-accredited.)
NABH accreditation requires the corresponding ethics committees to actively monitor regulatory trials.
In an interview with The Wire Science, Yashashri Shetty, an associate professor at the department of pharmacology, KEM Mumbai, and currently a member secretary of the KEM ethics committee, spoke about what on-site monitoring of clinical trials takes, the violations it can uncover, and how such monitoring can strengthen the conduct of trials in India.
The questions are in bold. The reporter’s notes are enclosed in square brackets. The text of the interview has been lightly edited for clarity.
What is the importance of an ethics committee in the context of clinical trials in India?
Yashashri Shetty: To protect the rights, well-being and safety of the participants. So in a nutshell, ethics committees are working for the participants. When they check the protocol, they see how complete the protocol is; are the objectives and the endpoints relevant to the clinical scenario in India and to the guidelines and regulations of India. So in a way, by giving critical comments on the protocol, they also try to protect the investigators and the sponsors as well.
Could you elaborate on how ethics committees also help protect investigators and sponsors?
For example, if there is just one coordinator doing multiple studies, we tell them there should be one coordinator per study so that the conduct of the trial is smooth, and there are fewer deviations in the study, and serious adverse events are reported on time. So in that sense we try to protect the investigators too – this is actually not needed from the ethics committee as such, but we go beyond our responsibilities.
One coordinator can do ten studies but that is not in the interest of the investigator itself because there will then be a lot of protocol deviations on the site.
All three KEM ethics committees are registered with the CDSCO and have voluntary accreditation. Is that right?
Yes, all KEM ethics committees are registered with both the Department of Health Research, which is required for academic studies, and with the CDSCO, which is needed for regulatory studies.
We also got international accreditation in 2009 from SIDCER. And in 2014, we got accredited from NABH.
Why did KEM ethics committees decide to go down the path of accreditation when it is not mandatory in India?
We had a lot of good members, like Dr Urmila Thatte, and many of the older members who had been in our ethics committees for a long time who said we should go beyond our standards. Accreditation brings with it a lot of streamlining of policies.
We also wanted someone to inspect us. It was not happening from the office of the Drug Controller General of India (DCGI), so we thought ‘why not go for an accreditation agency’, so they can come visit and review our entire conduct. That way, we can ensure the quality of our ethics committees ourselves.
We worked hard for two years for the international accreditation. We got all our papers in order, ensured that we were working as per the standard operating procedures (SOPs). Once we got accredited from SIDCER, it was very easy to go for NABH accreditation.
What kinds of changes did you have to make in order to get accredited?
Entire SOPs were changed. And we had to work as per the SOP. It was very rigorous. If the agenda has to be closed in three days’ time before the meeting, as per the SOP, the agenda had to be closed in that time period. You can’t bring papers in between. If meeting minutes have to go in 14 days, it means that minutes have to go in 14 days. If the comment letter [to the principal investigator] has to go within 14 days after the meeting, then it has to go in 14 days.
So it became very rigorous for the member secretaries. In KEM, most member secretaries have been from the pharmacology department and who are now well-trained in accreditation requirements. So it became very easy for seniors to handhold juniors.
What are the requirements of accreditation in terms of monitoring a trial once it starts?
Initially, we used to do passive monitoring. This means the principal investigator, or PI, used to give us continuing reviews in one year’s time. For that also we used to give reminders after 11 months, saying ‘next month, you are due for a continuing review’.
In the continuing review, they would report how many patients were screened, how many were recruited, how many adverse events and serious adverse events (SAEs) occurred, how many protocol deviations happened.
[Deviations from the approved trial protocol can happen in clinical trials, but when an investigator thinks that a deviation is likely to occur, they must have it prospectively approved from the sponsor, the ethics committee and the DCGI.]
So in a nutshell, we used to understand which site was not performing well. We used to then tag them for active monitoring. For example, if we saw that there were lots of SAEs in one trial area, we would go and monitor it. That was called ‘for cause’ monitoring.
But when we went for NABH accreditation, NABH told us that for regulatory studies it was compulsory to monitor all the sites. At that time, there were 13 sites conducting regulatory trials, so all 13 sites were monitored by us. That was a big change.
Does on-site monitoring require a different kind of training for the ethics committee members?
When we went for NABH accreditation, we got trained by people who had been trial site monitors, people who were in the trial sponsors’ data safety monitoring boards (DSMBs). There were two workshops focusing only on active site monitoring because the lay members of the committee also had to go to the sites. That was the initial thing we did before we started actual monitoring.
[Indian regulations require each ethics committee to include a layperson member who is not a scientific expert.]Could you walk us through how you go about doing the active monitoring of the sites? What are some general things you look at?
We have an SOP for on-site monitoring. If you see the SOP, it has a checklist. We see the master file initially and check what all the documents that were approved by the ethics committee are. Then we see if that master file is adequately available at that site. We check the screening log [which has the details of every participant who was assessed for inclusion in the trial], what the screening failures are, what the reasons for failure are, and whether the signatures were intact.
Then we see the log of delegation: who has been given what responsibility, and we check for the signatures accordingly. We check for signatures on informed consent documents. For example, if I’m in charge of obtaining informed consent, is my signature on the forms or has someone else signed?
Also, are the signatures right? For example if there is a paediatric patient, is there a signature of an LAR [legally acceptable representative]? If the patient and the LAR are illiterate, is there a signature of an impartial witness? Has the impartial witnessing been done correctly?
Sometimes, we actually call a patient in the trial to see if she has understood the study as a totality. Why did she participate? What are her reasons for participation? Why does she want to continue? Was there any coercion factor when she came?
We also randomly check whether the protocol’s inclusion and exclusion criteria were followed. For example, if there are 20 patients in a trial, we pick at least five sample patients at random and check if the inclusion was exactly as per the protocol. Then we look at the case record forms (CRFs) [a CRF is a document to record data on each trial participant as per the protocol] to find out if there are any adverse events that have not been informed by the sponsor.
We also check if all the SAEs were reported as per mandated timelines. Was the initial reporting done on time within 24 hours and later a follow up report was done in 14 days? Were the DCGI and sponsor adequately informed about the SAE?
At last, we interview the principal investigator and we try to find out how much he actually knows about the protocol. Many of the PIs are not aware of the protocol because they are not involved at all. This could be because the study coordinator, and not the PI, might be doing everything, or because the PI is doing a lot of studies.
In a nutshell, we try to see exactly what is happening at the site. We also see the internal monitoring report of the sponsor, which the PI has to send us. Many times they don’t, but because we know the protocol, we know when a DSMB inspection is included in the protocol. So we ask them – did that inspection happen, was the report created, and if yes, please submit the report. Then we see that if there is some finding from the sponsor, was that corrected at the site or did they make the same mistakes again when we did the monitoring.
According to your published findings, violations of the informed consent process appear to be the most common. Could you give us a few examples of these violations and tell us why these violations keep happening in your opinion?
The major violations we have seen have to do with the impartial witness.
[According to the New Drugs and Clinical Trials Rules 2019, “if the trial subject, his or her legally acceptable representative, is unable to read or write, an impartial witness should be present during the entire informed consent process who must append his or her signature to the consent form”. This impartial witness is supposed to be an independent person not involved in the trial herself nor can she be influenced by the people involved.]Usually, one study coordinator takes another as an impartial witness, but that is actually not acceptable. The PI signing for the impartial witness should also not be done. These are common mistakes.
Then there are columns for signatures for the investigator, participant, LAR and impartial witness. Some people understand that all four have to be signed, not that only the signatures that are eligible in that particular case are required. For example, even if the patient is an adult, we sometimes see an LAR’s signature. They don’t understand that that signature is not required.
[A legally acceptable representative, such as a family member, can sign on behalf of an adult participant only if the participant herself is unable to meaningfully provide consent.]One reason these violations keep happening I think is because of the kind of training in good clinical practices (GCP). The PIs usually do their GCP training through online courses conducted by the US National Institutes of Health – but the NIH training does not give you real-life examples. So even when we find violations and give them a warning and ask them to do their training again, they redo the NIH training and submit a certificate. That’s a problem.
Is there no Indian GCP training course online?
No, there’s no Indian GCP training that is readily available for you to do anytime you want to do it. So usually, people go for the NIH training because this is a handy training that’s available online. You read the material for six hours, you answer the questions and a certificate is generated.
You also found protocol deviations during your on-site monitoring. Could you give some examples?
In one study, for example, the inclusion criteria said that if the patient is in severe chest pain, the PI will take the signature of an LAR. This is because that patient needs immediate treatment and is not in a condition to give signature himself. Signatures take a lot of time. You have to make them understand the informed consent document. But we saw that all patients had been signing.
We asked the PI about this, but he said that such a thing is not written in the inclusion criteria. We showed him the protocol, which said that in a situation where a patient is in distress, LAR signature can be taken. So the PI was not aware of the protocol.
Another example is that at one site the approved sample size for that site was 30. But the PI took 60 patients because he did not remember the sample size that had been approved. He had a lot of patients, so he kept recruiting.
When the ethics committee observed this violation, we said the 60 patient data can’t be included either in your publication or in your report to the DCGI. So that data could not be used by the sponsor.
You also found instances of either the absence of source documents or improperly stored documents.
One of the GCP principles is that you have to maintain the privacy and confidentiality of documents in order to maintain the privacy and confidentiality of the patients. So we say that all documents have to be under lock and key.
But what has happened is that in some sites, multiple studies are happening and all the files for all trials were kept in one cupboard. That is one scenario we’ve found. In many scenarios, there is no lock and key. Anybody can access the files. The coordinator of study 1 who is not involved in study 2 also has access to study 2’s files. So there is no maintenance of privacy and confidentiality.
What happens once you observe these violations?
We actually send the investigators a monitoring report, saying that these are our findings and these are the corrective actions which are required. Then they give us the corrective actions. Sometimes we also give them a warning letter when there’s a gross violation of GCP.
For example, if there are lots of informed consent process violations, we say that this is not acceptable and you’re not trained and we submit a warning letter. Later, say if we go for monitoring twice, and if we don’t find any rectification, we have halted studies as well. We’ve not taken protocols [proposed trials] from that PI for six months as punishment, saying that we will not accept trials from your site because your site is not trained.
The problem is that a few PIs are untrained and are not willing to get trained, so these violations keep happening. But they are also the ‘big people’ in their respective branches, so sponsors approach them. When we’ve found that the PI is not equipped, that they’re not ready to train themselves or change themselves, we have not allowed studies from them later.
Have you found that this kind of active monitoring has improved conduct, and if yes, could you give examples of the kinds of improvements you’ve seen?
One improvement we’ve seen is regarding keeping files under lock and key. Now if you see, in our sites, documents are under lock and key. There are loggers for all biologicals or drugs that are to be kept in negative-degree temperatures, so a temperature log is maintained properly because they know we can come any time and check for all these.
Protocol deviations and SAEs have been reported rigorously from the sites. Many times, when we went back to the same site again, the PIs were aware of the protocol. There is fear also from the PI’s side because they need to submit our letter of findings to the sponsor as well.
So if there are many violations, their image in front of the sponsor is also tarnished and these are some very big people in their branches. They don’t want that, and we see a lot of improvement because of site monitoring.
If on-site monitoring has so many benefits, why don’t more ethics committees do it?
Many of the ethics committees are not equipped to do on-site monitoring and many don’t want to monitor because they don’t want to give any more time to the ethics committee. They say ‘we’re attending the meeting, that in itself is a big thing that we’re doing’. Money is an issue, too. External members are still paid but internal members aren’t paid at all. So why should they spend time on monitoring as well when they’ve already spent so much time doing the review?
How do KEM committee members solve these problems?
In our case, it is compulsory for all members to monitor at least one site. We have ratified this.
What is the response that you have received from the PIs and sponsors to your active monitoring?
Overall, the feedback has been positive.
Do you think there should be a regulatory mandate requiring active on-site monitoring?
It can be made compulsory but it is very tough to actually execute such monitoring. I don’t think it’s acceptable to make something compulsory that can’t be executed. But for-cause monitoring should be compulsory. That is, if you find a reason to monitor, you should monitor.
What are the implications of letting violations in clinical trial conduct go undetected, for a layperson?
One is that the rights and the well-being of the patients are compromised. For example, if adverse events and serious adverse events are not reported on time, we don’t come to know what is happening at that site, and then the patient is harmed, according to me.
If protocol deviations are unnoticed, we also don’t know what the data credibility is of that site. If a drug is supposed to be tested in a certain format that’s given in the protocol, and you don’t do that – that is called protocol deviation – and if it is unreported, we won’t know the credibility of the data.
I’ll give you an example of a trial that was conducted during COVID-19. There were lots of biomarkers that had to be analysed in a study with severe COVID-19 patients, but the biomarker analysis was supposed to happen in another city, and since there was no transport available in Mumbai as well as no flights available because of the lockdown, this biomarker analysis did not happen at all.
[Biomarkers, or biological markers, are proxy characteristics of your body that can be measured objectively to yield insights into how your body is doing. For example, your blood pressure or heart rate are biomarkers that can give doctors an indication of the condition of your heart. The level of D-Dimer, a small piece of protein in the blood that results from the breakdown of a blood clot, has been used as a biomarker to predict the severity of COVID-19.]It’s only after each biomarker analysis that the second dose was supposed to be given to the patient. But the second dose was given without this analysis. Fortunately there were no deaths, although the drug was approved despite these violations. So we don’t know what the data credibility is.
For a layperson this is very important because in practice, when I recommend this drug, I will look at these biomarkers. But it wasn’t looked at in a clinical trial setting itself, so the implication is huge.
Also read: Serum Institute Fracas Exposes Loose Ends of India’s Clinical Trial Machinery
Did your onsite monitoring reveal this issue? And what course of action did you take then?
We found it during a monitoring visit, and we actually questioned the PI and [others at] the site. Later, as we could not get correct answers, we wrote to the sponsor and the DCGI office to reconsider the KEM data. But after sending letters to the PI, sponsor and the DCGI three times, we did not get any reply from them. The study got completed and the drug was approved for emergency use.
Why did you decide to publish the findings of your on-site monitoring?
Initially, when we thought of doing this monitoring, we thought there would be papers on it, so we started doing a review of the literature. But we were unable to find anything from India or abroad. Whatever KEM ethics committees do as an activity that has enriched their functioning, we have tried to publish that data. We think our findings can help others.
What have been your top lessons from active on-site monitoring?
Our main lesson is that you can’t blindly trust an investigator. You can’t feel that ‘this is a good site or a good investigator so there can’t be any deviations’. We’ve found that good sites also have lots of deviations. And unless you monitor, you can’t know what that site is doing.
I think every site should be monitored at least once. For example, if a PI is doing ten studies, you don’t have to monitor all ten sites, but if you monitor even one study, you’ll be alarmed, and she’ll do a good job moving forward.
The reporting for this article was supported by a grant from the Thakur Family Foundation. The foundation did not exercise any editorial control over the contents of the article.
Shreya Dasgupta (@ShreyaDasgupta) is an independent science writer based in Bengaluru, India. Her work has appeared in Mongabay, Nature, BBC Earth, Smithsonian.com, New Scientist, Ensia, and other publications.