Photo: Alena Shekhovtcova/Pexels.
Precisely two months after India’s national drug regulator approved it for emergency use in “clinical trial mode” on January 3 – a decision that came under severe criticism and some confusion – Bharat Biotech has announced the first set of preliminary results from the phase 3 clinical trials of its Covaxin vaccine candidate.
Covaxin is a whole virion inactivated vaccine against novel coronavirus infections (COVID-19).
The principal result is that Covaxin has been found to have an efficacy of 80.6%. This figure is quietly encouraging, and is in line with the company’s previously published reports based on data from the phase 1 and phase 2 trials, which indicated that Covaxin was suitably immunogenic and had a tolerable safety profile.
News of the reported efficacy is tempered by the fact that it is based on an interim analysis of the data, based on just 43 ‘events’. That is, among the 25,800 people who were recruited and randomly, but equally, split between the vaccine and placebo groups, there were 43 cases of COVID-19 at the time of this analysis.
According to the trial protocol, specified in the company’s registration on the American clinical trials registry, the primary efficacy analysis in the study would be triggered once the cohort had 130 symptomatic COVID-19 cases between 42 days and 12 months after the second dose of the vaccine or placebo had been administered.
The trial protocol provided for a formal interim analysis at two time-points: upon reaching one-third (43 cases) and two-thirds (86 cases) of the primary target.
The March 3 press release issued by Bharat Biotech is thus the headline result of the first interim analysis, triggered by the occurrence of 43 cases of COVID-19 in the study cohort. (The researchers confirmed them with RT-PCR tests).
Understanding vaccine efficacy
The announcement yesterday also included that the 43 cases were split as 36 in the placebo group and seven in the vaccine group. What the press release didn’t tell us, however, was the denominators for the two groups. Since the randomisation was a straight 1:1, i.e. the two groups were of equal size, we can assume that there were 12,900 people in each group.
Vaccine efficacy is, at its heart, a straightforward mathematical concept. The virus’s ‘attack rate’ in the placebo group is effectively the ‘background rate’, or the infection rate we infer from real-world data. The vaccine’s efficacy is the percentage by which the ‘attack rate’ in the vaccine group is lower, weighted by the extent to which this effect is the result of the vaccine.
In the present case, since the vaccine and placebo groups are of the same size, we can replace the ‘attack rate’ with the actual number of cases:
Efficacy = 100 – 7/36 = 80.6%
The total number of cases of COVID-19 observed is relevant here because if the analysis is based on too few cases, there is a high probability that any differences observed are due to chance, and not attributable to the vaccine. This is also why the Covaxin phase 3 trial protocol allowed for a primary outcome analysis only after 130 cases had occurred.
Second: yesterday’s announcement of the preliminary results didn’t mention a confidence interval (CI) for the 80.6% efficacy figure. The CI describes the range across which a given figure can vary, as a result of possible variations in the data used to calculate it. All clinical trials that report data based on large cohorts do so along with CI data as well. If the latter is missing, please search for it.
Alternatively, it’s also possible to work out the approximate 95% CI in this case using some standard statistical tools: 56.4% to 91.3%. (The 95% CI is calculated such that there is no less than a 1-in-20 chance that the CI does not include the data’s true mean. It’s a qualitative safeguard.) This is a wide range. In other words, it’s very likely that Covaxin is protective but the small number of ‘events’ (43) based on which the researchers are calculating efficacy means that we can’t be sure about exactly how efficacious it is.
To compare, the 95% CI for the efficacy of some other vaccines that have been approved are described in this table:
Clearly the 95% CI can be much tighter, relative to the ‘background rate’. In qualitative terms, such tightness allows researchers and others to be more confident that the trial’s results demonstrate the vaccine’s effects and not the effects of any other events or chance. And until this is the case with the Covaxin phase 3 trial data as well, we must step cautiously.
Other details that are not yet available are the demographics of patients who were recruited. The intention in the protocol was to recruit men and women aged 18 to 99 years, but we do not yet know to what extent the actual recruitment may have departed from this plan. We also do not have data on the side effects experienced by the trial participants. No doubt these details will emerge when the trial results are published.
Also read: India Approves Two Vaccine Candidates, But Let’s Not Pretend Everything Is Okay
Conclusions
Nonetheless, the interim results are encouraging. For now, we must regard the 80% efficacy figure as just a preliminary estimate. As more data emerges and the researchers repeat their analyses with more cases, the efficacy may well turn out to be higher. And whatever it may be in the final analysis, we will also be able to assess it along with a more precise and reliable estimate of the efficacy and more data on potential side-effects.
The release of this preliminary data prompts an important question: What was the point of the Drug Controller General of India’s conditional approval for Covaxin, on January 3, 2021, when he and the Central Drug Standards Control Organisation must have known that preliminary data from the phase 3 trial would be available soon?
The controversy over what was widely seen as premature approval achieved little for India’s vaccine industry – even as it gave rise to unnecessary and avoidable doubts and speculation. The approval could even have contributed to widely reported vaccine hesitancy among healthcare workers slated to receive either Covaxin or Covishield, without a choice, in the first phase of the country’s vaccination drive.
Reuters has also reported legal action in Brazil to stop their government’s plan to purchase 20 million doses of Covaxin.
Hopefully, the interim results announced yesterday will allay some of the underlying concerns. And if this positive result is confirmed in later analyses with tighter CIs, and the researchers also report a good safety profile, the recent controversies could become water under the bridge. However, that two important institutions responsible for enforcing drug and vaccine standards failed to follow due process should not be forgotten.
India is fortunate to have a talented pool of scientists and successful pharmaceutical companies. We just need to have faith in the process and allow our scientists to do the right thing.
Dr Jammi Nagaraj Rao is a public health physician, independent researcher and epidemiologist in the UK.
This work by The Wire Science is licensed under CC BY-ND 4.0
