Fake blood in test tubes labelled ‘COVID-19’, March 2020. Photo: Reuters/Dado Ruvic/Illustration.
Bengaluru: In May, the Indian Council of Medical Research (ICMR) conducted a countrywide COVID-19 seroprevalence survey. Although the full details of this study haven’t yet been made available in the form of a paper, peer-reviewed or otherwise, ICMR has already announced a follow-up study as well.
In July-August, the municipal corporations of Delhi, Mumbai, Ahmedabad and Pune released some of the results of their respective citywide seroprevalence assays, which they had conducted together with different research institutes.
Seroprevalence surveys are useful to understand the extent to which a given population has contracted a specific infectious disease. These surveys use tests that detect the presence of antibodies, which the body’s immune system would have produced to fight the pathogen – in this case the SARS-CoV-2 virus.
Upon infection, the body initially produces a class of antibodies called immunoglobulin M (IgM), which has a short lifespan and is eventually converted – seroconverted, in technical parlance – to immunoglobulin G (IgG). The presence of IgG is used as an indication by proxy of past exposure to the virus.
“Virus specific IgGs are produced in a later stage during the course of a virus infection, as a person slowly starts recovering. Antibodies by themselves are produced as a pool of different types, among which a fraction constitute the neutralising antibodies that prevent the entry of the virus inside host cells,” Upasana Ray, a senior scientist of infectious diseases and immunology at the Indian Institute of Chemical Biology (IICB), Kolkata, said.
“These are the ones that we need to look for if we want to determine whether an individual is protected or not.”
As on August 7, 2020, ICMR had approved 12 IgG test kits that could be used for seroprevalence surveys. One of them is the Kavach IgG ELISA kit, which ICMR developed in May and licensed to Zydus Cadila, Ahmedabad. Anecdotal reports suggest it’s one of the most commonly used kits in the surveys conducted thus far.
Since the results of seroprevalence surveys indicate what measures policymakers should institute to control the national COVID-19 epidemic, the kits play an outsize role in determining our course of action.
To this end, the Department of Biotechnology (DBT) India Consortium for COVID-19 Research recently compared the performance of three such test kits. The study was led by Shinjini Bhatnagar, a professor at the Translational Health Science and Technology Institute (THSTI), Faridabad, and Gagandeep Kang, an expert on viral infections in children who quit as the head of THSTI a month ago. They uploaded a preprint paper describing their findings on August 14 (the ‘published’ version has since become available here). The kits they tested were:
1. RBD ELISA – developed at THSTI
2. Diasorin LIAISON SARS-CoV-2 IgG CLIA – developed in Italy, meets the UK Medicines and Healthcare Products Regulatory Agency (MHRA) criteria, has emergency-use approval from the US Food and Drug Administration
3. Zydus Kavach ELISA
The RBD ELISA and Diasorin CLIA assays attempt to detect the presence of the virus’s spike protein in a given sample. The RBD ELISA assay can detect antibodies against the receptor binding domain – the part of the spike protein that binds to the ACE2 receptors in humans. The Diasorin assay detects antibodies against S1/S2 regions of the spike protein. There are no details in the public domain about which viral proteins the Zydus Kavach assay looks for.
The researchers evaluated these assays on 379 COVID-19 positive samples and 184 COVID-19 negative samples.
If there was a test kit designated the ‘gold standard’ to detect SARS-CoV-2 IgG antibodies, researchers could use it to see how accurately new kits could detect these antibodies in comparison. But since there is no such kit, the team followed the UK’s MHRA guidelines to define positive and negative samples.
“We used a positive reference panel which included samples from participants who had more than 20 days of duration of illness following RT-PCR positivity,” Bhatnagar said.
That is, members of the consortium considered samples collected more than 20 days after the onset of illness among symptomatic people and those from people who tested positive for COVID-19 by RT-PCR as being the positive samples for the antibody tests to work with. The COVID-19 negative samples had been collected before September 2019.
When they compared these kits, the team found that they were less sensitive than what the internal studies of the organisations that developed them had claimed.
Also read: What You Need to Know Before Trying to Make Sense of Sero-Survey Results
The researchers tested the samples using Zydus Kavach and Diasorin LIAISON assays, following instructions provided in the kit, and interpreted the results as positive or negative accordingly. When the results were ambiguous, or weren’t defined by the manufacturer, they considered them to be ‘negative’. With the RBD-ELISA kit – they defined a criterion and interpreted the results as positive or negative based on it.
The authors then compared how many of the samples had been correctly identified by each of these assays as ‘positive’ or ‘negative’, and calculated their respective sensitivity and specificity.
When ICMR announced that it had built an antibody testing kit, its press release claimed a specificity and sensitivity of 100% and 98.7%, respectively. Following an investigation by The Wire Science, the kit’s makers at ICMR and the National Institute of Virology, Pune, admitted that the kit was 97.9% specific and 92.37% sensitive.
In the current study, the researchers found the kit was 99.5% specific and only 75.7% sensitive.
Also read: What Are Sensitivity and Specificity?
The published version of the researchers’ paper offers one explanation for why this difference could be so – because of the different methods with which the kits were validated: “The article reporting internal validation of Zydus Kavach ELISA reported a sensitivity of 92.37% based on samples that were positive in a micro-neutralisation assay as against 75.7% that we see in our study using sera collected [more than] 20 days post-RT-PCR positivity.”
RBD ELISA was found to be 84.7% sensitive. The Diasorin kit, which the maker had claimed was 97.4% sensitive, was found to be 82% sensitive. The team couldn’t check its specificity due to the limited availability of negative samples.
“While unlikely, the difference” in the Diasorin kit’s performance “could be attributed to false positives in our RT-PCR assay,” the paper noted.
About 14% of the samples obtained from asymptomatic people might have had low antibody levels and therefore might not have produced IgG antibodies at the time of sample collection. However, Bhatnagar, Kang and their team-mates also compared the three assays head-to-head to check which performed best. They found that the RBD ELISA and Diasorin CLIA kits could identify more positive IgG samples than Zydus Kavach.
“The performance of RBD-ELISA being on par with an MHRA-evaluated Diasorin assay and slightly better than the Zydus Kavach assay is encouraging,” Bhatnagar said. “This definitely provides an alternative to the existing options in the country, particularly in a scenario where more sero-surveillance studies are needed to understand the evolution of the pandemic better.”
THSTI has transferred the RBD ELISA assay for commercialisation.
“This study will definitely help improve the determination of seroprevalence, but in its own specific way,” Ray said.
That is, for example, RBD ELISA could help detect “the percentage of people who are recovering, have recovered or were asymptomatically infected and are producing IgG antibodies against the virus.” However, it may not be able to identify newly infected people who aren’t producing IgG antibodies yet.
Ray noted that another important aspect of developing antibody test assays is to ensure there is no cross-reactivity. This is the phenomenon whereby the kits misidentify antibodies against other common viruses to be antibodies against SARS-CoV-2. This will yield false positive results.
So studies conducted to develop these assays will specifically have a panel of negative samples known to contain antibodies against specific viruses other than the one that the test intends to detect. For example, in this case, you could conduct the test on a panel of SARS-CoV-2 IgG-negative samples that have antibodies against H1N1, dengue virus, rhinovirus, etc. – and see which antibodies the assay mistakes for the ones against the novel coronavirus.
“We did not specifically report cross-reactivities in this article. We used pre-pandemic samples that are assumed to have antibodies from other viral infections but not antibodies to SARS-CoV-2,” Bhatnagar said. “If a false positive result is obtained, it may be due to other viruses, including seasonal coronaviruses, which the individuals may have encountered.”
The assay also detects only possibly neutralising IgG antibodies – the kind produced only after some days of viral infection. But if the body doesn’t produce them, a kit might say ‘negative’ even as the individual may really be infected.
Also read: No COVID-19 Test Is 100% Right, so Their Errors and Results Are Both Important
So, Ray said, if a survey aims to study the spread of a virus in a population, it should employ kits that detect non-neutralising antibodies as well – and researchers should study the link between these molecules and a COVID-19 infection more.
“During a survey or diagnosis, the objective could be different, and the diagnostic methods should be adjusted to the required agenda. So, availability of different methods using different rationale would provide the chance to choose an assay type most suitable for a certain study,” Ray said.
For example, in a population in which the virus’s prevalence is low, an assay with high specificity could be more useful, since it will return fewer false positives. Also, SARS-CoV-2 IgM assays can detect persistent infections in the population – whereas SARS-CoV-2 IgG assays can detect those who are recovering or have recovered.
The Indian Council of Medical Research, the National Institute of Virology and Diasorin didn’t respond to requests for comment.
Note: The headline of this article was changed at 3:40 pm on August 31, 2020.
Joel P. Joseph is a science writer.