Focus on Official Statistics Makes Covid-19 Worse, But Look Better

​​​​​​​(Part I of “Statistics and the Pandemic”)

Mark Twain said, “There are three kinds of lies: lies, damned lies, and statistics.”  Too often, the pandemic has unnecessarily allowed scope for the sort of popular suspicions reflected in Twain’s bon mot.

Statistics are in fact a critical component of the fight against Covid-19.  Their use ranges from judging the efficacy of different vaccines to judging the performance of different governments.

But throughout the pandemic, comparisons across countries have focused too much on the wrong statistics. The problem is worse than impeding voters’ evaluation of governments’ performance.  The focus on the wrong metrics has given some political leaders a strong incentive to under-react to the pandemic, to suppress testing for example, and thus has arguably contributed to the loss of millions of lives.

Per capita cases

The official global death toll from Covid-19 stands at 3.3 million.   One sees tendentious media reports comparing fatalities across countries or states.

For professionals analyzing the data, the more information the better.  But for the purposes of the general conversation, the choice of which sort of statistics to emphasize is consequential.  Everyone has limited bandwidth and in this age of mistrust of experts and mainstream media, prominence must be given to statistics that are as informative and reliable as possible.

Counts of countries’ officially recorded infections and deaths have been the most prominently featured.  The first thing one needs to do is pretty obvious: divide by population, to express the numbers in per capita form.  It is not very enlightening to hear that the United States has suffered more cases, in absolute terms, than the Netherlands.  Or India than Peru.

Undercounting

But even reported on a per capita basis, official counts of infections and deaths are tremendously inaccurate.  Specifically, they are usually gross under-estimates of the true numbers.  Carriers of the virus are not counted as infected unless they are tested or even hospitalized.  Victims are not counted as Covid-19 deaths unless a doctor puts that on their death certificates.

Estimates of the extent of the undercount of infections vary.  The same with deaths. The World Health Organization this week reported an official mortality under-count of 50 % in the European region and 60% in the Americas. The Economist magazine’s model of excess mortality calculates that the true number of deaths attributable to Covid-19 globally is currently in the range of 7 – 13 million, that is, about three times the standardly reported official account.

The estimated undercount varies widely across countries.  It is often greater under autocracies — a factor of 5 in Russia, for example.

The undercount also tends to be more extreme in lower-income countries.  Egypt, for example, has 13 times as many excess deaths as the number officially attributed to Covid-19.  The lowest-income countries may not even be able to count total deaths at all.  In that case, estimates of excess deaths can only be inferred.  The Economist’s model shows an undercount factor of 14 in Sub-Saharan Africa.

For richer countries, the overall undercount is smaller, estimated at 17 % for OECD members.  In the US, the estimated undercount was running at only 7 % in March-April.  The American undercount is much smaller in 2021 than it was last year.  In the early days of the pandemic, President Trump said there were only 15 American casualties and congratulated himself for doing such a good job (February 26, 2020).  As late as July 2020, the professional estimates of true infections ran 2-to-6 times higher than the official US count.

Of course, even accurate measures of infections or deaths would be just the starting point for evaluating the effectiveness of public measures to combat the virus.  One would need to control for other determinants, such as population density and age distribution, before passing judgment on the quality of government crisis management.

That official reports of verified cases and attributed deaths understate the true count is no secret.  But the focus on official reports, which only count infections according to positive coronavirus tests or deaths according to what is on the death certificate, does more damage than merely understating the gravity of the problem.  Worse, it gives politicians an incentive to reduce testing and suppress bad news, e.g., by encouraging doctors to list co-morbidities (diabetes) or complications (pneumonia) as the cause of death rather than Covid-19.  Since testing, social distancing, and vaccination — most of it voluntary — have been so necessary for fighting the coronavirus, whatever undermines them causes deaths.

In other words, the focus on official infection and death statistics has made Covid-19 look better than the reality, but has made the reality worse.

Some statistics that are more informative

It would be better if the media and popular discussion paid more attention to some other measures that are more informative and also less vulnerable to political influence.

The positive coronavirus test rate as a percentage of all tests administered is much more instructive, compared to positive results as a percentage of the entire population. India’s recent positivity rate has been a very alarming one in four. (It is as high as one in two in West Bengal. In contrast, the US positivity rate has recently fallen below 3 %.) If President Modi had focused on the positivity rate earlier this year, perhaps he would have known not to prematurely declare victory against the virus. As it was, he facilitated super-spreading mass political rallies and religious celebrations.

If the positivity rate had received more attention than the reported infection rate, it would have constructively given governments an incentive to increase testing.  Instead, the focus on verified infections as a percentage of the population gave governments a dangerous incentive to reduce the extent of testing.  When Donald Trump was president, he was acutely sensitive to this incentive: “When you test, you find something is wrong with people. If we didn’t do any testing, we would have very few cases,” May 14, 2020. (He claimed that the US was doing more testing than most others, but the claim was completely false.)

Trump was not alone among leaders. Brazilian President Jair Bolsonaro limited testing for the sake of appearances and actually cut back when Covid-19 cases accelerated sharply.  Others who have sought to downplay the seriousness of the virus include Mexican President Andrés Manuel López Obrador, Russian President Vladimir Putin, U.K. Prime Minister Boris Johnson, and now-deceased Tanzanian President John Mugufuli.

In the current phase, the vaccination rate can be a revealing measure of government progress. Israel has now vaccinated 57 % of its population, Chile 41 %, the US 39 %, and the U.K. 34 %, whereas most EU countries have only vaccinated around 15 % of their populations, Brazil and Mexico 9 %, and India 3 %.  Within the US, the vaccination rate ranges from above 60 % in New England states to a low of 33 % in Mississippi.

Epidemiological models often focus on the parameter R, the rate of exponential spread of the virus (or, after turning the corner, the rate of exponential decline).  Unfortunately, data availability does not allow reliable estimation of R.

Positivity rates, vaccination records, and excess mortality calculations have their own limitations as criteria of how bad things are.  But they are generally more illuminating than the figures that are most widely cited.

Statistics can be used to mislead; hence Mark Twain’s cynicism.  But that is a reason to delve a bit more deeply into statistics, not to avoid them altogether.