An excellent article in the Financial Times, published this past Monday, illuminates one major challenge of estimating a vaccine campaign’s success: population data are not always reliable. Health reporter Oliver Barnes and data reporter John Burn-Murdoch explain that, in several countries and smaller regions, inaccurate counts of how many people live in the region have led to vaccination rate estimates that make the area’s vaccine campaign look more successful—or less successful—than it really is.
Why does this happen? It’s actually pretty challenging to get a precise count of how many people live somewhere. Think about the U.S. Census, for example: this program attempts to count every person living in the country, once every ten years. But it may miss people who don’t have a straightforward living situation (like college students, the incarcerated, and people living in shelters); it may have confusing messaging that discourages some people (like undocumented residents) from filling out the necessary form; and some people may simply choose not to give information to the government.
When the Census is inaccurate, the inaccuracies ripple out to different government analyses—including analyses of how many people have been vaccinated. Here’s a quote from the Financial Times article:
“The average person would be surprised that governments don’t know how many people are actually in the country,” said Stian Westlake, chief executive of the UK’s Royal Statistical Society. “But this great unknown can cause a whole host of data glitches, especially when responding to a health emergency.”
The Financial Times provides several examples of these data glitches leading to incorrect vaccination estimates.
- In England: Overestimates of the unvaccinated population, based on data from the U.K.’s Health Security Agency, suggest that case rates are lower among unvaccinated Brits than they actually are.
- In several EU countries: Underestimates of the senior population lead to vaccination rates inaccurately suggesting that over 100% of certain age groups in Ireland, Portugal, and other countries have received at least one dose of a vaccine.
- In Miami, Florida: A number of ZIP codes have senior vaccination rates that appear to be over 100% of seniors, due to retirees (who do not have permanent residence in Florida, and therefore aren’t counted in the state’s population) getting vaccinated in Miami during their winter vacations.
Incorrect vaccination rates can cause issues for public health agencies leading vaccine campaigns, the Financial Times reports. If you think you have vaccinated 100% of seniors in your county due to population underestimates, you likely aren’t looking out for the seniors who in fact remain unvaccinated—leaving those seniors still vulnerable to COVID-19.
At the same time, data glitches can provide fodder for anti-vax groups. “Worst of all, anti-vaxxers and Covid deniers feed on the daylight between reality and the incomplete data we currently have as evidence of a grand conspiracy or bureaucratic incompetence,” Jennifer Nuzzo, epidemiologist Jennifer Nuzzo told the Financial Times.
I recommend reading the Financial Times article in full. But you can also check out this Twitter thread from John Burn-Murdoch for more highlights: