Now Reading
Tests Used by BMC in Mumbai Had Many False Negatives. Don’t Be Surprised.

Tests Used by BMC in Mumbai Had Many False Negatives. Don’t Be Surprised.

A health worker in PPE uses his mobile phone to record details of residents during a COVID-19 check-up campaign in Mumbai, July 22, 2020. Photo: Reuters/Hemanshi Kamani.

Bengaluru: Recent reports of the rapid antigen tests for COVID-19 having 60-65% false negatives in Mumbai have alarmed many – but didn’t surprise experts.

On July 24, the city’s Brihanmumbai Municipal Corporation (BMC) launched ‘Mission Universal Testing’, a programme through which the civic body aims to test every person in Mumbai for the novel coronavirus. Pursuant to this goal, it plans to purchase one lakh antigen testing kits that will be made available in private hospital and labs as well, according to India Today.

There are three types of diagnostic tests for COVID-19: the RT-PCR test, the antigen test and the antibody test. The RT-PCR test looks for viral DNA in a given sample. The antigen test looks for certain proteins belonging to the virus, used as a proxy for the virus’s presence in the sample. The antibody test looks for antibodies that the antigen would have provoked and which would be present in the blood.

The antibody test can’t detect COVID-19 in its early stages as the body develops antibodies only one to three weeks after an infected person has shown symptoms – so it is used to detect a past infection. This is the type of tests the BMC used for its recently concluded seroprevalence survey, which indicated that nearly 7 million of Mumbai’s residents, and over half of all residents of the city’s slums, could have already had COVID-19.

On the other hand, the RT-PCR and antigen tests are used to test whether an individual is currently infected. The RT-PCR test is more accurate and more time-consuming, infrastructure-intensive and requires trained personnel. The antigen test is less accurate, yields results in 15-30 minutes and can be performed at the site where the sample is collected instead of shipping samples to a lab.

These advantages render antigen tests an attractive candidate from the point of view of speed – but speed isn’t the only concern. Part of the picture here is that the novel coronavirus spreads very fast, which has three implications.

First, a test must also screen people effectively, perhaps even at the expense of returning a few false-positive results. This way, according to Tinku Thomas, a professor and head of the department of biostatistics at St John’s Research Institute, Bengaluru, a test can pick up “at least almost all the positives” for isolation and/or treatment, and these people can be rechecked later with RT-PCR tests to make sure.

The second consequence is related to public health and awareness. A test that quickly tests people at the expense of taking more time to yield more accurate results could also yield more than a few false negative results. That is, infected people who test ‘negative’ may be led to believe they’re not actually in infected and become complacent, and not be so careful about wearing masks when stepping out, washing their hands whenever possible and/or taking extra precautions when meeting people with comorbidities.

Third, we must finally add the virus’s prevalence in a given population.

Understanding how all these factors come together requires some simple math.

Also read: What Are Sensitivity and Specificity? A Handy Reference

Ideally, a test kit should identify an infected person to be ‘positive’ (true positive) and a healthy person to be ‘negative’ (true negative). However, no kit is perfect. There are chances a test will find a healthy person to be ‘positive’ (a.k.a. false positive) and an infected person to be ‘negative’ (false negative).

The measure of how many true positives a test kit detects from among all the infected individuals is called its sensitivity. This number defines how sensitive a test is to the presence of the virus in an infected person. For example, if a test claims 90% sensitivity, the kit will be able to find 90 out of 100 actually infected people to be ‘positive’. The remaining 10 will falsely test negative. The lower a test’s sensitivity, the more false negative results it will return.

The measure of how many true negatives a test kit detects from among healthy individuals is called its specificity. This number defines how specifically the test can find a healthy person to be ‘negative’. If the test claims 95% specificity, it means the kit will be able to find 95 out of 100 healthy people to be ‘negative’. The other five would be missed out as false positives. The lower a test’s specificity, the more false positive results it will return.

In addition, the accuracy of a test also depends on how many people in the population are infected at a given time, i.e. the prevalence. As prevalence increases, the chances of a positive test being a true positive increase – but the chances of a negative test being a true negative decrease.

While RT-PCR and antibody tests are over 70% and 90% sensitive, respectively, the sensitivity of a rapid antigen test kit is about 50%. As on July 23, 2020, the Indian Council of Medical Research (ICMR) had approved three rapid antigen test kits for use in India. The details of two of them are as follows (those of the third, COVID-19 Antigen Lateral Test Device, aren’t available in the public domain):

Table 2: Sensitivity, specificity, false positive rate and false negative rate of ICMR-approved rapid antigen test kits.

Thomas said the number of false negatives can be high even when the specificity is a 100% because “false negatives are actually positive cases that are not getting picked up by the test, due to low sensitivity.”

Rahul Siddharthan, a professor of computational biology at the Institute of Mathematical Sciences, Chennai, elaborated on this possibility: “One can have a high specificity, where most of what is detected as ‘negative’ is indeed ‘negative’, while also having a high false negative rate. If 50 out of 100 of your samples are actually positive, and you predict ‘negative’ for all of them, your true negative rate is 100% – i.e., you have 50 true negative and no false positive predictions. But your false negative rate is also 100%. Your sensitivity … is zero in this case.”

On July 23, India Today quoted a doctor in its report as saying the rapid antigen test kits used in Mumbai’s seroprevalence survey were 50% sensitive. So the false negative rate – the chances of a true positive case being missed – is also 50%. “I would say a reported 60-65% false negative rate is not very far off from a claimed 50% sensitivity,” Siddharthan said.

The COVID-19 Ag Respi-Strip, developed by the Belgium-based company Coris Bioconcept, has sensitivity as low as 30.2%. So a false negative rate of close to 70% is possible.

Then there is also human error – that enough virus particles weren’t collected in the swab for the test to detect an infection. This could contribute to false negatives as well.

Also read: Why Experts Are Calling ICMR’s New Advisory on Antigen Tests Unreasonable

If we performed a rapid antigen test in a municipal ward with 100,000 residents, of which 10,000 are infected, using a kit that has 50.6% sensitivity and 99.3% specificity, we would miss 4,940 people who are actually infected as false negatives.

Calculating the number of false positives and false negatives possible in a population of 100,000, of which 10,000 are infected. Image: Author provided

This is because, of the 10,000 people that are actually infected, the test is sensitive enough to find only 5,060 (i.e. 50.6%) to be ‘positive’. The remaining 4,940 (i.e. 49.4%) that are infected will incorrectly test ‘negative’. This outcome risks the people who falsely tested ‘negative’ – especially if they were asymptomatic and/or whom contact-tracing efforts missed – believing they don’t have the virus and lowering their guard when interacting with other people. So it becomes critical to confirm whether they are truly ‘negative’ or falsely ‘negative’ using a more sensitive test.

According to ICMR’s guidelines, all those who may have falsely tested negative with a rapid antigen test are to be retested with an RT-PCR test. Many experts have said this is an inefficient strategy, considering the fact that COVID-19 is bound to have a low prevalence in India’s vast population, so the chances of a large number of test results being false negatives are high. “One might as well conduct the RT-PCR tests for all,” Thomas said. “Adding on the burden of retesting false negatives is not economical.”

Joel P. Joseph is a science writer.

Scroll To Top