In last week’s newsletter, I gave a shout-out to the Salt Lake County Health Department, which posted this novel vaccination data on Twitter:
The post drew a lot of attention in the COVID-19 data world, including with readers of the COVID-19 Data Dispatch. (Shout-out to the reader who sent me some bonus analysis of vaccinations by Zodiac element!) Unfortunately, additional research into the Salt Lake County Health Department’s data has shown me that the agency’s analysis might not be particularly robust—and I feel it is my journalistic duty to share this with you.
Here’s the deal. In order to calculate vaccination rates by Zodiac sign, you need two things: vaccinations organized by birthday (your numerator), and the overall population organized by birthday (your denominator). Health departments can easily access the numerator, as it is standard for people to provide their birthdays along with other basic demographic information when they get vaccinated.
But the denominator is trickier. The average U.S. public health department doesn’t have access to the birthdays of every resident in its jurisdiction; some information might be available from a large hospital system or primary care network, but it wouldn’t be comprehensive. So, for an analysis like the Salt Lake County agency’s, a researcher needs to find a substitute.
In this case, the researchers used estimates of Zodiac sign representation in the entire U.S. population, apparently calculated in 2012. Not only are these numbers based on birthdays across the entire country (which could be pretty different from the birthdays in one Utah county!), they’re almost ten years old. There’s a lot of distance between these estimates and vaccination numbers among a 2021 Salt Lake City population.
The public health workers acknowledged that their analysis is “not super scientific” in interviews with the Salt Lake Tribune. Still, the widely-shared Twitter post itself could do with a few more caveats, in my opinion.
For more on the issues with the Salt Lake County department’s analysis, see this Substack post by Christopher Ingraham.