An illustrative photo: Maksim Goncharenok/Pexels.
New Delhi: Days after grabbing headlines with its COVID-19 “vaccine for the world”, AstraZeneca is facing tricky questions about its success rate that some experts have said could prevent it from being by US and EU regulatory agencies anytime soon.
Several scientists have raised doubts about the robustness of results showing the vaccine candidate was 90% effective in a group of clinical trial participants who accidentally received a half dose followed by a full dose (instead of the planned two full doses).
“All we have to go on is a limited data release,” Peter Openshaw, a professor of experimental medicine at Imperial College London, told Reuters. “We have to wait for the full data and to see how the regulators view the results,” he said, adding that US and European regulators “might possibly take a different view” from each other.
The British drugmaker claimed on Monday that its experimental vaccine – developed with Oxford University – prevented around 90% of participants in the group that received a half dose (in Britain) from developing COVID-19, while the corresponding figure was 62% in a group in Brazil that received the doses as originally intended.
These figures are both above the 50% efficacy required by US regulators. Europe’s drug regulator has said it will not set a minimum level of efficacy for potential vaccines.
Also read: The Public Must Be Aware of the COVID-19 Vaccine’s Limitations
But at the heart of these concerns is the fact that the trial’s most promising result of 90% comes from a sub-group analysis – a technique many scientists say can produce spurious readings. “Sub-group analyses in randomised controlled trials are always fraught with difficulties,” Paul Hunter, a professor of medicine at Britain’s University of East Anglia, said.
In particular, according to him, such analyses increase the risk of “type 1 errors” – whereby an intervention is considered to be effective when it is actually not.
This is in part because the number of participants is greatly reduced in a sub-group – making it harder to be confident that a finding is not just down to chance differences or similarities among participants.
“In order to have faith in the results,” Hunter said, any sub-group analysis “should be sufficiently powered” with large numbers of volunteers to take readings from.
Only 2,741 volunteers were in the sub-group that yielded the 90% efficacy read-out. This is a small number compared to the tens of thousands of participants of the clinical trials for Pfizer-BioNTech’s and Moderna’s vaccine candidates, with reported efficacies in excess of 90%.
‘Devil in the detail’
AstraZeneca said both independent data safety monitors and the UK regulator had reviewed the half-dose incident and had allowed the trial to continue, adding that the regulator publicly confirmed there was “no concern”.
“We are in discussions with regulators around the world to evaluate these findings and we look forward to the publication of the peer-reviewed results, which has now been submitted to a journal,” a spokesperson added.
The US regulator, the Food and Drug Administration (FDA), has not commented on AstraZeneca’s vaccine trial results. The European Medicines Agency said on Thursday it would “assess data on the efficacy and safety of the vaccine in the coming weeks once they have been received from the company”.
The regulatory process has nonetheless been clouded, according to experts, who have also noted crucial gaps in the data AstraZeneca has made public thus far.
“The devil is in the detail,” said Danny Altmann, a professor of immunology at Imperial College London. “We’re trying to assess really quite complex trial designs on the basis of little press releases.”
Beyond headline efficacy rates, AstraZeneca’s data release gave little for scientists to work on. It did not say how many infections occurred in the sub-group, for example, or in the group that got two full doses – or in fact in the placebo group.
“A lot of questions are left unanswered,” said Morgane Bomsel, an expert at the French National Centre of Scientific Research. “We are under the impression [AstraZeneca] are selectively picking out the data.”
According to the New York Times, other crucial information is also missing. The company hasn’t shared details as to which group the 131 symptomatic COVID-19 cases that were identified belonged to – those who in the half-dose/full-dose trial or the full-dose/full-dose trial.
There is also confusion due to the company’s decision to pool results from two differently designed clinical trials, in Britain and Brazil, and calculate an average based on them. This is a break from standard practice in reporting the results of drug and vaccine trials, according to the New York Times.
“I just can’t figure out where all the information is coming from and how it’s combining together,” Natalie Dean, a biostatistician and an expert in vaccine trial design, told the newspaper. On Twitter, she said AstraZeneca and Oxford “get a poor grade for transparency and rigour when it comes to the vaccine trial results they have reported.”
Menelas Pangalos, the AstraZeneca executive in charge of research and development, told Reuters on Monday that the company had not intended for any participants to receive the half dose. British researchers running the trial there had meant to give the full dose initially to volunteers, but a miscalculation meant they were mistakenly given only a half dose. He described the error as “serendipity,” allowing researchers to stumble onto a more promising dosing regimen.
With Oxford and AstraZeneca unable to explain why the half dose was more effective than the full dose, outside experts and analysts are starting to doubt the credibility of the results altogether. “I think that they have really damaged confidence in their whole development program,” Geoffrey Porges, an analyst for an investment bank, told NYT.
‘A number of variables’
Moncef Slaoui, chief scientific adviser for the US government’s vaccine programme Operation Warp Speed, also highlighted gaps.
He said no-one in the subgroup that got the initial half dose was older than 55 years – suggesting the regimen’s efficacy in the crucial older age groups remains unaccounted for in the interim data.
In the group that received a correct full dose followed by a full dose, he noted, older people were included.
Such concerns, and the possible consequences for the speed of regulatory approval, helped AstraZeneca shares hit their lowest level since April on Thursday, having fallen 7% since the company released the data on Monday.
By contrast, Moderna has rallied 22% since releasing its vaccine trial data on November 16, and Pfizer and BioNTech are up 6% and 14% respectively since announcing their successful data on November 9.
“There are a number of variables that we need to understand, and what has been the role of each one of them in achieving the difference in efficacy,” Slaoui told a briefing on Tuesday.
“It is still possible that the difference (in efficacy) is a random difference,” he added. “It’s unlikely, but it’s still possible.”
Also read: COVID-19 Vaccine: Why Does the Serum Institute of India Have a Head Start?
Inexpensive, but also inexperienced
The AstraZeneca-Oxford vaccine is expected to cost only a few dollars (i.e. a few hundred rupees) per dose, unlike the ones Pfizer and Moderna are producing. It can also be stored in normal refrigerators for months, also unlike the other two candidates.
While Pfizer’s and Moderna’s candidates use the more advanced mRNA vaccine technology, the AstraZeneca candidate uses a modified version of a chimpanzee common cold virus to provoke an immune response to the coronavirus.
The company estimates it can produce three billion doses next year – enough to vaccinate nearly a fifth of the world’s population.
But according to the New York Times, the company is “inexperienced” when it comes to manufacturing vaccines. Even before the “chaotic release of results”, the company had been under scrutiny for its handling of the testing process.
In September, clinical trials were paused worldwide after a participant fell ill but the company did not promptly announce the decision in public. The company also provided details about the nature of the illness on a private conference call with investors rather than disseminating the information more widely.
If the vaccine proves to be efficient, it would prove to be a boost for India. The vaccine is already being mass-produced in the Serum Institute of India (SII) in Pune. With a $200-million investment in place, the SII had already produced over 2 million doses in late October. The institute told DW that it has the ability to ramp up production to 10 million doses per month should the candidate be authorised.
If the vaccine is approved, SII and the Indian government have committed to reserving half the company’s stock for India, while the other half will be supplied to low-income nations through GAVI, a vaccine alliance that sponsors immunisations for low-income nations.
“If the trials are successful, we will make the product available in as many countries as possible, including India. When a vaccine is developed, we plan to sell it for around €12 (Rs 962) per dose,” said SII’s CEO Adar Poonawalla.
(With Reuters inputs)