Kiran Mazumdar Shaw, chairperson and managing director of Biocon Ltd. Photo: Reuters.
On July 13, this year I questioned claims made via a press release and a TV news channel’s panel discussion that Bengaluru-based Biocon Ltd.’s monoclonal antibody drug, itolizumab, was a “life-saving breakthrough” that could help people with severe COVID-19 infections.
At the time the trial data had not been published, and I drew on whatever Ie could glean from the entry in India’s Clinical Trials Registry, and from the Biocon press release.
My verdict then was that if the drug had any potential value in treating severe COVID-19 illness, it had been let down by poor science. Only a much larger randomised controlled trial, I said, could settle the question.
Since then, two things have happened. First, the US biotechnology company Equillium, with which Biocon has an exclusive partnership to develop and market itolizumab, announced on November 25 that it was abandoning plans to conduct phase 3 trials for itolizumab with 800 patients.
In hindsight, this may well be a great pity because of the other development.
The research team that conducted the original inadequate trial on itolizumab, with 30 patients, uploaded the paper corresponding to the trial on December 2, with a fuller account of what they did.
Of the 10 authors listed in this pre-paper, four are from Biocon Limited. Of the others, four are also listed as principal investigators in the trial protocol; one Dr Shashank Joshi is not, although he was a key participant in a discussion on TV hosted by Rajdeep Sardesai.
The last author is a surprising new entrant: Randeep Guleria, the director of the All India Institute of Medical Sciences, New Delhi. He was previously not publicly known to have been associated with the trial – neither in the protocol as set out in the trial registry nor in earlier publicity about the emergency use authorisation of the drug.
Most of all, the 41-page preprint paper leaves us precisely where we were back in July with respect to the evidence – or lack of it – for itolizumab’s efficacy vis-à-vis the management of cytokine release syndrome, which is sometimes a feature of severe COVID-19.
As set out in the paper, the trial was an open-label phase 2 randomised controlled trial of itolizumab plus standard care among 20 patients versus standard care alone among 10 patients. It had been conducted in four hospitals, two in Mumbai and two in Delhi.
The primary outcome was 30-day all-cause mortality – as it was in the original protocol I reviewed in July. However, the trial’s entry in the register today lists no less than five additional primary end-points, all of which are surrogate outcomes (that is, indicators or proxies of the real outcomes). To be measured at periodic intervals over the 30-day duration of the follow-up, these are:
* Deterioration in lung function defined as stable oxygen saturation, or normal arterial oxygen level without additional inhaled oxygen
* Need for intubation
* Need for respiratory support (high flow nasal oxygen, non-invasive ventilation, or intermittent mandatory ventilation)
* Duration of mechanical ventilation in those who needed intubation
* Change in inflammatory markers in blood tests.
But despite the expansion of the primary endpoints from one to six, the target sample size remained the same – 30.
We are not told which primary endpoint, or combination of endpoints, the Biocon researchers used to calculate the target sample size.
Also read: The Worrying Ways in Which Biocon’s Itolizumab Trial Was Deeply Flawed
Indeed, there is no explicit mention of the size of the difference in outcomes between the two groups that the researchers eventually considered to be clinically significant, nor do they present a power calculation.
For unknown reasons, the first five patients were apparently part of a “staggered initial dosing”. After each patient was given the drug, they were monitored for 24-48 hours before the next patient was administered the drug.
After this phase, subsequent patients were randomly allocated in a 2:1 ratio to the active and control arms of the trial. The paper does not tell us the outcome for these first five patients.
The CONSORT flow diagram on page 7 tells us that of the 22 patients randomised to the active treatment arm, two patients did not receive a complete dose because of an infusion reaction, and that they were removed from follow-ups.
The authors considered them “unevaluable” and their data is not part of the outcome results, but they were included in the safety analysis. This is a remarkable omission for reasons that will become clear soon.
The best supportive care consisted of a mix of oxygen, antibiotics, steroids, hydroxychloroquine, low molecular weight heparin and vitamin supplements. itolizumab administration was preceded by 100 mg of hydrocortisone, administered intravenously.
The results
The primary endpoint, of mortality at 30 days, showed that three of 10 patients in the control arm and none of the 20 in the itolizumab arm, died. This difference – 30% mortality versus 0% – the authors say is statistically significant. The three deaths in the control group occurred on days 4,5, and 12.
For brevity, I shall not deal with the other numerous endpoints, other than to note that the surrogate markers that were measured at successive points should really have been presented as survival curves rather than as a binary outcome at each time point with multiple statistical tests.
Surprisingly, for a trial that had a clear binary outcome, the results are presented in a confusing manner. Instead of describing the event rates, the relative event rates and the numbers needed to achieve successful treatment, the authors have provided p-values with a 95% confidence interval. This is not the norm.
Also read: Why the Way We Use Statistical Significance Has Created a Crisis in Science
As for the two patients who were randomised initially to the treatment arm but were subsequently replaced – their fates are tucked away on page 24.
Recall that two patients randomised to the active treatment arm did not complete the first dose due to an infusion reaction and that these patients were replaced. One of them recovered and was discharged two weeks later. The other patient, sadly, “developed further complications of COVID-19 related ARDS and died nine days later after discontinuation, and the event was deemed not related to the study drug.”
Eliminating these two patients from the follow-up analysis raises serious problems when considering what might happen when a new treatment is used in clinical practice. This is why the best practice is to perform an intention-to-treat (ITT) analysis. Here, once randomised, patients stay in their allocated groups for the purposes of analysis even if their care has departed from the protocol for that group for whatever reason.
In a strict ITT analysis, where these two patients would have been included, the results would have read, “One of the 22 patients in the active treatment arm and three of 10 in the control arm died”. And the final analysis would have looked like this:
The implications
The authors are more guarded in the discussion and conclusion portions of their paper – more so than they were in the TV panel discussion in July, when the same results were doled out as breakthroughs and life-savers.
The results are described as “encouraging”, and the authors acknowledge that “there is a need to replicate these findings”. However, in the same sentence, they reveal their desire for a positive result by qualifying the need to repeat the study “either through additional, larger clinical trials or post-marketing surveillance studies”.
Larger clinical trials are simply not the same as post-marketing studies. The real need is for a large, well-designed, double-blind randomised controlled trial of the kind that Equillium had in mind.
So it’s a real pity that Equillium has cancelled its plans for an 800-patient trial – and because Biocon Limited’s considerable resources have not been deployed into organising a large trial by themselves.
Between July, when the Drug Controller General of India granted emergency use authorisation for itolizumab, and today, India has had its COVID-19 case load grow by just under 9 million, and the recorded deaths due to COVID-19 by more than 100,000. There was no shortage of COVID-19 patients for a trial that might have definitively answered the itolizumab question.
But there was – and still remains – a decided lack of willingness on the part of big and rich pharmaceutical companies to fund proper, scientifically robust, statistically meaningful clinical trials. To repeat what I said back in July: “India’s pharmaceutical industry cannot become world-class on the back of shoddy clinical trials”.
Dr Jammi Nagaraj Rao is a public health physician, independent researcher and epidemiologist in the UK.