A healthcare worker in protective gear squeezes the sweat out of his face masks as he takes a break from collecting swab samples for rapid antigen tests, Ahmedabad, July 24, 2020. Photo: Reuters/Amit Dave.
Two weeks ago, I argued that Biocon Limited’s claim of a breakthrough in the treatment of severe COVID-19 with their drug Itolizumab was not justified. It was based on a single small, flawed and inadequate clinical trial. Nevertheless, the Drugs Controller General of India (DCGI) granted the company an ’emergency use authorisation’ to treat acute respiratory distress syndrome (ARDS) mediated by cytokine release syndrome (CRS), due to COVID-19. The decision has come under heavy criticism for lax regulatory standards.
Further, on July 27, India’s National COVID-19 Task Force decided to not include Itolizumab in their management guidelines. In a press briefing, Indian Council of Medical Research (ICMR) director-general Balram Bhargava said Biocon’s trial was too small for the drug to be considered reliable.
The right approach to this setback would have been a) to retract the older claims, based on the inadequate trial; and b) to design and undertake a larger phase III randomised controlled trial (RCT), preferably in many centres in India and elsewhere. That would have allowed the company – if the results were indeed favourable to Itolizumab – to seek regulatory approval in jurisdictions other than in India.
Instead, Biocon Limited appears to have dug its heels in. Its response to the scientific criticisms of their claims has been ordered along two lines.
The first has been to repeat the argument that Itolizumab has been shown to work in the treatment of psoriasis, that the inflammatory process in psoriasis is similar to that in CRS in COVID-19 patients, and that its use for psoriasis means it has an established safety profile. For these reasons, Biocon Biologics’ global head of regulatory affairs Sundar Ramanan has argued that a larger phase III study was not necessary. Biocon’s corporate communications head Seema Ahuja has endorsed this line on Twitter.
“By effectively controlling hyper-activation of immune system, Itolizumab prevents morbidity n mortality related to cytokine storm. Its Mechanism of Action in COVID is similar as in Psoriasis (approved post Phase III) Additional Phase III trial for CRS, COVID thus not required” https://t.co/4kmu6MsBYC
— Seema Ahuja (@SeemaAhuja1) July 29, 2020
This is at best a spurious argument, based on understanding disease mechanisms and drug actions at the molecular level but erroneously translating them into statements of efficacy that ‘therefore the drug must work’. The uncomfortable fact is that [our understanding of the cytokine response in COVID-19 is far from perfect, and fondly held theories are often found wanting when tested in rigorous RCTs.
The second tack has been to offer more data to satisfy regulators, and in particular to convince the task force of Itolizumab’s efficacy. At first glance, this appears counterintuitive to their conviction that a phase III trial is not necessary, and almost like a grudging response to calls for a larger phase III RCT. However, the “real-world evidence” Biocon is offering could make matters worse. Let’s understand why.
“Biocon intends to publish Real-World Evidence in peer-reviewed journals. We have always claimed that Itolizumab is for the right patient at the right time, and is not for all patients with #COVID19.” @sundarramanan provides facts on #itolizumab Read > https://t.co/nb8xSjBTSj pic.twitter.com/mHHavrzFTn
— Biocon (@Bioconlimited) July 29, 2020
On July 13, in a press release, Biocon provided a taste of early results from a Cuban trial of Itolizumab. “Of 76 patients treated with the drug, 79% of severely ill patients were discharged from the ICU, while moderately ill patients showed a reduction in the rate of progression of the disease”. No further details were available.
It’s unclear whether it was the same study but on July 30, Cuban researchers uploaded a pre-print paper that reported the results from an Itolizumab trial. This study was also poorly designed, and only adds to the noisy mess of claims and speculations. It was a non-randomised study in which 19 elderly patients infected in a nursing home outbreak were treated with Itolizumab. The researchers compared their outcomes with a group of control patients selected from a larger database.
The statistical presentation of the data in this paper leaves much to be desired. The lack of random allocation renders any conclusions about the drug’s effect quite suspect.
So, what is ‘real-world data’? It amounts essentially to data obtained in the course of the clinical treatment of patients. Such data will include documentation of each patient’s clinical symptoms and lab measurements, treatments administered, clinical course and the final outcomes.
When this information is collected in a database and the database is analysed as a whole, researchers can find interesting associations between particular features among patients and the treatments that may have been tried, and the outcomes. They are useful to generate hypotheses and ideas; however, crucially, such studies can’t establish a causal link between a specific treatment and a specific outcome. Ascertaining such links requires an RCT.
This is why Biocon’s proposal to gather data from patients treated with Itolizumab outside a clinical trial on an off-label basis raises the spectre of bias. Clinicians treating patients with Itolizumab will be asked to submit data on their patients. How can we know doctors won’t reply only with their success stories? If they don’t include every patient given the drug, the results will become vulnerable to the Texas sharpshooter fallacy, which is a form of selection bias: when one omits one’s mistakes and presents only examples of one’s best work to claim a level of infallibility that does not reflect the truth.
Would doctors who may have used Itolizumab be so crass as to omit cases where the outcome was not as desired? Of course they would. Doctors are human, too. This is why the CONSORT standards for reporting RCTs require that every patient recruited into a trial is accounted for in the final write-up. Transparency is crucial to scientific research.
And any such real-world data that Biocon might now collect may not include cases that recovered without the use of Itolizumab.
The fact is that the only way to reliably attribute successful treatment to a specific drug is to conduct a properly designed and large-enough RCT. There are those who have argued that the ongoing coronavirus pandemic is an emergency, that we can’t therefore afford the standards of scientific research that have protected us thus far from errors, biases and fraud. This line of reasoning is dangerous, ill-informed and ill-serves patients, doctors and the public.
Dr Jammi Nagaraj Rao is a public health physician, independent researcher and epidemiologist in the UK.