People wearing protective face masks leave the Chhatrapati Shivaji Terminus railway station, amidst the COVID-19 outbreak, in Mumbai, September 22, 2020. Photo: Reuters/Niharika Kulkarni.
Some surprising results from the second round of Mumbai’s seroprevalence survey have been in the news. The survey, carried out in late August, found that 45% of slum dwellers surveyed and 18% of non-slum dwellers had detectable levels of antibodies to SARS-CoV-2. What is remarkable is that the value for slum areas is about 10% lower than in the previous survey, covering the same wards in early July. Meanwhile, antibody levels amongst non-slum dwellers increased only marginally between the surveys, even though cases and deaths, driven by non-slum areas, continued apace throughout July and August.
These results have been reported as “good news” – but “perplexing news” might be more appropriate.
Let’s recall that the goal of seroprevalence surveys is to estimate prevalence – i.e. the extent of prior COVID-19 infections. These should not, in general, decrease over time. So how could prevalence in the surveyed wards fall significantly in a space of six weeks? Does this mean that earlier accounts of Mumbai’s epidemic were built on flawed data and need to be rewritten? If earlier estimates of prevalence were too high, then we would have to revise estimates of the city’s infection fatality rate (IFR) – the fraction of people infected with COVID-19 who die – upward. This would not be good news.
It turns out that the new data does not change Mumbai’s story dramatically. Our previous understanding of disease levels, fatality and uneven spread remains largely valid. But the survey does teach us some important lessons. Let’s dig a little deeper.
Possible explanations
Three possible explanations for the apparent drop in prevalence in slum areas come to mind.
1. The drop is real. Between the seroprevalence surveys, there was a genuine fall in the fraction of slum-dwellers exposed to COVID-19 in the surveyed wards.
2. The drop is apparent and is a consequence of sampling errors. The people tested in one or both surveys were not representative of the slum population of the wards.
3. The drop is apparent and is about the tests. The antibody test used fails to pick up waning levels of antibodies in individuals infected some time ago.
Any or all of these could have occurred. However, when we look more closely, it seems unlikely that the first two explanations could play a major part. (More detailed versions of the arguments to follow, along with calculations and references, are here.)
A real fall in prevalence on the scale observed would require major demographic changes in the surveyed wards. The most plausible would be a large migration of people not exposed to COVID-19 into the wards. While there has been some return of migrant workers to the city, many must have been exposed to COVID-19 before they left in May. Moreover, the return would have to be on a startling scale to affect the prevalence significantly enough.
Several news reports suggested that sampling errors could cause an apparent drop in prevalence. When full details of the second survey are available, it will be possible to assess this explanation more carefully – but at face value it does not stack up. The sampling approach used in the surveys appears carefully designed to avoid biases. The sample sizes were more than sufficient, and from the details given by the authors after the first survey, major under- or overestimation of prevalence is unlikely.
We are thus left with the final explanation: the tests are increasingly not picking up past infections. Let’s examine this possibility more closely.
Waning antibodies and decreasing sensitivity
It is now well known that antibodies to COVID-19 wane quite rapidly. This does not necessarily imply a rapid loss of immunity: the relationship between antibody levels and protection from infection is complicated and only partly understood for COVID-19. However, it does affect seroprevalence survey results.
Whether an individual exposed to COVID-19 is picked up in a seroprevalence survey depends on their antibody levels, but also on the test being used. Antibody responses may be weaker in individuals with mild or asymptomatic disease, and different tests may be more or less sensitive to low levels of antibodies.
For a given test, a certain proportion of individuals with detectable antibodies shortly after infection will later “serorevert” – i.e. they will no longer have detectable antibodies some time later. Thus, the sensitivity of an antibody test – its likelihood of picking up past infections – drops as an epidemic continues. The rate and extent of this waning depends both on the test and the population being surveyed.
Now, what do we know about the “Abbott Diagnostics Architect N-protein based test” used in Mumbai’s two seroprevalence surveys? Here, we get lucky. Researchers have studied the Abbott kit with the goal of answering precisely the questions we are interested in. Studies in New York and Brazil have attempted to measure the waning sensitivity of this test in order to arrive at reliable prevalence estimates from repeated surveys in Manaus and São Paulo.
Brazilian approach, Mumbai data
We would really like to know the “true” prevalence we would obtain if the test was 100% sensitive and if this sensitivity did not diminish over time. To estimate this, we need to know two things: the proportion of people who serorevert and the speed at which this happens. Although we don’t know these for Mumbai, we can try a range of values from the Brazilian study. We can also vary assumptions about early spread in Mumbai prior to the first seroprevalence survey. Some resulting estimates of true seroprevalence at the time of the second survey are summarised in the following table.
All the estimates tally with the story of a dramatically uneven spread between slums and non-slum areas. The lowest values come from assuming later disease spread, and relatively slow and infrequent antibody-waning. The higher values are obtained by assuming the opposite: earlier disease spread, and frequent, rapid antibody waning. It is reassuring that these estimates accord with earlier prevalence estimates using cases and the first seroprevalence survey data.
What does all this imply about Mumbai’s naïve IFR ignoring fatality undercounting? We get estimates in the range 0.09% to 0.14%, with the lower values of IFR corresponding to the higher values of prevalence, and vice versa. Taking underreporting into account – we find that the city’s IFR would lie somewhere in the range of 0.16% to 0.30%, close to values estimated from Mumbai’s age-structure, fatality data, and first seroprevalence survey.
§
Although movement of people and sampling errors might explain some part of the apparent drop in prevalence between Mumbai’s seroprevalence surveys, it is likely that the major cause is waning sensitivity of the test used. Correcting for this gives estimates of prevalence in line with other evidence, and with what we expect based on the city’s fatalities.
So Mumbai’s story has not changed dramatically with the preliminary results of the second survey. They do, however, suggest that we interpret survey data cautiously, especially when an epidemic spans a long period of time (already about seven months in Mumbai’s case). This could be relevant to Delhi too: the third Delhi seroprevalence survey in the first half of September found lower seroprevalence despite an average of about 1,250 daily new cases throughout August.
A practical conclusion is that it is vital for seroprevalence surveys to measure the waning sensitivity of different antibody tests by repeatedly testing individuals who have recovered from COVID-19.
The author thanks @CovidSerology for drawing his attention to studies in New York and Brazil on the Abbott test kit.
Note: This article was edited at 2:05 pm on October 6, 2020, to clarify that studies on the Abbott test kit happened in New York and Brazil, and not just Brazil as stated first.
Murad Banaji is a mathematician with an interest in disease modelling.