COVID-19 Data Dispatch

Tag: interview

  • Remission Biome could represent a new paradigm in patient-led research

    Remission Biome could represent a new paradigm in patient-led research

    Screenshot of the Remission Biome website.

    I have a new story out in National Geographic this week about a growing area of research connecting the gut microbiome—the diverse community of microorganisms that live in our digestive systems—with myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS), the chronic disease that often occurs after viral infection and has many commonalities with Long COVID. (Here’s a non-paywalled link to the story.)

    Two recent papers, both funded by the National Institutes of Health, point to specific differences between the gut ecosystems of ME/CFS patients and those of healthy controls. The new studies built on previous research in this area, but utilized larger patient cohorts than typical ME/CFS studies. Their findings provide avenues for better diagnosing and treating ME/CFS patients, as well as people with Long COVID who meet the criteria for ME/CFS. (Some studies suggest about half of Long COVID patients fall into this category.)

    Continued gut microbiome research could identify specific bacterial supplements that help alleviate ME/CFS symptoms, along with new drugs that target specific changes in these patients’ metabolisms and immune systems. But many people with ME/CFS and Long COVID aren’t waiting for the clinical trials; they’re experimenting with the supplement and diet changes that are already available.

    One big project in this realm is Remission Biome, a study by two ME/CFS patients who were working scientists before their symptoms became disabling. Patient-researchers Tamara Romanuk and Tess Falor both have experienced “remission events,” in which their ME/CFS symptoms faded after taking bacterial supplements. When they met on Twitter, they started a project attempting to recreate those events—but it quickly grew into a much larger effort to understand how the microbiome interests with post-viral illness symptoms.

    I talked to Romanuk and Falor for my National Geographic story. We discussed how to design a study in which the scientists are also the patients, how their project builds on big data in microbiome research, communicating with the patient community, future plans for Remission Biome, and more. Since most of the conversation didn’t make it into my story, I wanted to share it here.

    This interview has been lightly edited and condensed for clarity.

    Betsy Ladyzhets: I wanted to start by asking how you all came to do this project, specifically replicating remission events that you’d experienced in the past, and how that intersects with other research that’s been done in this area of ME/CFS and the microbiome.

    Tamara Romanuk: I had always been planning on trying to replicate the event… It had been in my mind constantly to try to recreate the experience. I actually did, at one point, take another course of amoxicillin [an antibiotic] but I didn’t do anything else. I didn’t do the probiotics. I didn’t do the sort of the full suite that I was that we’re proposing here [in the Remission Biome protocol].

    So, it’s something I would have come to on my own regardless. But when I met Tess and she told me that she had had a similar event, it seemed like, okay, this was something we were probably going to do in the future together. It was just a matter of when. I think we must have talked about it even within the first weeks of meeting each other.

    Tess Falor: It was, 10 days [after we met]… I had an interesting experience, in that that I didn’t realize that it was the antibiotics that might have done this [alleviated symptoms]. So back when I had my event in 2009, I went gluten-free and started the antibiotics at the same time. And two days later, after I had my major remission, I accidentally ate gluten. And then I felt worse the next day. So for 13 years, I’ve been assuming that it was going gluten-free that made the difference. And I never consider that I could recreate it because I’ve been gluten for that whole time.

    It wasn’t until I met Tamara last fall that I realized that, wow, this [remission event] actually could have been the antibiotics. So when that came up, then, Tamara had this idea to try to recreate it. If it was actually the antibiotics that did it, could we create this really extreme remission event?

    BL: I see. In terms of identifying the antibiotics, the probiotics, and the other treatments that you’re using in this study, how did you arrive at this protocol that y’all are following?

    TR: Yeah, great question. When we decided to recreate it, the plan started as, “let’s take the antibiotics, and let’s throw in some probiotics, and let’s see what happens.” But we’re scientists, so it morphed very, very quickly into something much, much bigger.

    Where Tess and I went in our minds was directly to the theory and the models that would have given rise to a phenomenon like this. Instead of starting at a protocol, we went, “this is our pet theory [about why remission occurred], and if our pet theory is right, how do we work backwards and recreate the protocol?” One of the really interesting things about doing it from that angle was that most of the stuff that I had initially thought I would include I actually threw out very, very quickly. The probiotic that I thought we were almost for sure going to use has ended up being, “oh, that actually might have stopped my event from continuing rather than promoting it.”

    And we’re working in an age where there’s some really new cutting-edge databases and information sources out there. I wouldn’t have actually been able to do this type of protocol development a few years ago, even. [For example], we knew that we wanted to manipulate tryptophan metabolites in the gut. And so we needed to find strains of bacteria that were involved in tryptophan metabolism. And then we also knew that we wanted really specific metabolites to be produced in the gut and get to the brain. We were able to go into a database and actually follow that chain, identify specific strains of bacteria that would do exactly what we wanted in terms of gut and brain metabolites, and then trace it back to probiotic manufacturers. That’s pretty phenomenal. And that wouldn’t have been possible two years ago.

    BL: Yeah, that’s incredible that you could just follow it all the way through like that.

    TF: Absolutely. I’ll also mention that we’ve gotten advice from experts, too. Three people who study the microbiome gave us specific advice, and it all kind of converged in the same ideas, the same strains. So that was cool, too.

    BL: Actually, it’s interesting that you mentioned tryptophan, because that was one of the processes identified in the recent papers [the two studies that were the focus of my National Geographic story] as well. I think the main one they looked at was butyrate.

    TF: Yeah, those are both things that we are thinking about as part of our hypotheses. When those papers came out, we were like, “wow, this is really cool timing.” We read the papers, and found [their findings] line up with what we’re thinking. And we’ve already been thinking about this for months.

    TR: There’s a really neat tie-in here in terms of the tryptophan metabolism. Because initially, my theory had been surrounding Robert Phair’s idea of the “Metabolic Trap,” which is, of course dependent on tryptophan. But the [remission] event itself was also incredibly unique because it was a bit psychedelic. Colors were brighter, smells were smellier, the world was amazing, we felt overwhelming gratitude. I tell people that it was a little bit like a mix between doing MDMA and psilocybin mushrooms. But without the hallucination. It was just this overwhelming change in my mood.

    So I was thinking about tryptophan, serotonin, the same receptors that might actually get activated during a psychedelic trip. It really seemed that this entire pathway—from tryptophan to serotonin, to some good and some bad molecules like kynurenine, which can be both pro- and anti-inflammatory, and then quinolinic acid, which is absolutely pro-inflammatory in the brain. When you start at tryptophan, you keep going along this pathway, and whatever path it ends up following, you get different neurotransmitter activity going on. [The remission] was a microbiome event, but it’s really a microbiome-mediated event that’s occurring in the brain.

    BL: That makes sense, yeah. Because it’s the microbiome that impacts these metabolic processes that then impact what’s happening in the brain, right?

    TR: Absolutely.

    BL: In terms of tracking what happens when you do this protocol, what are you using to study the changes in your gut and the further progression that you were talking about?

    TF: Tamara knows more details than I do, but I’ll just say we are measuring a ton of biomarkers.  Everything from specific composition of the gut, to measures of the immune system, like cytokines, and measures of what’s going on in the brain.

    TR: We probably have about a thousand metabolites that we’re gonna track. When you consider that we’re doing a lot of these pre[-trial], hopefully during as well as post, it really adds up. It’s actually one of the most exciting parts about the project, for Tess and I. We both have an explicitly systems thinking approach to science, and we love big data. This is something that actually really excites us, we’re going to be able to really dig in.

    All of these tests really work in concert as well. The immunogenetics angles are really key for us. And Tess and I have some unique, similar genetic backgrounds, so that’s going to tie in. Then tracking neurotransmitters: actually tracking tryptophan, tracking serotonin, tracking kynurenic acid and quinolinic acid, as well as their ratios. And all of these metabolites.

    BL: I see. Yeah, I feel like that will be really interesting to look at all the interactions between these different things. I know you mentioned that you’ve consulted with folks on the protocols, are there other things that you’re doing to maintain the safety of the experiment?

    TR: Absolutely. So we’re working with one of my personal GPs [general practitioners], and she’s kind of acting as our GP safety liaison. She will be on hand if anything strange happens. But really what we’re doing in terms of safety is we’re testing ahead of time. We’re making sure we don’t have leaky gut, we’re making sure we don’t have a compromised blood-brain barrier, or making sure that we don’t have certain pathogenic bacteria in our system that could explode if they weren’t affected by the amoxicillin—and lead to a massive, very adverse reaction.

    The main tests we’re doing there is from a company called Cyrex, they do these amazing immune tests… And then we’re doing tests like the GI effects from Genova, which will actually tell us if we have high levels of any pathogenic or potentially-pathogenic bacteria in our guts, before we actually start. If any of [certain concerning biomarkers] turn up as being really high for one of us, then we will take some time out, try to correct that specific defect and then proceed again.

    BL: That makes sense.

    TF: We do have a clinician who has a lot of experience with these specific tests, and specifically with the gut, who’s helping us. In addition to Tamara realizing it was a good idea to do all this pre-testing, she recommended it, too. So she can take a look at our results, and give us her perspective.

    TR: One of the things we want to do there is, we want to actually figure out a very simplified testing protocol, which we can suggest to people to do for themselves before they do this [experimenting with supplements]. Because we want to actually bring this work to people who really need it, but we also want to make sure they’re safe. Hopefully, all of the testing that we do will help us arrive at a couple of biomarkers, which we can then tell people, “Look, if you test these one or two or three things, then you can do this without worrying so much about having side effects or adverse consequences.”

    Right now, it would cost the regular person probably about $2,000 to test everything that we’re testing, in terms of making sure all these levels are safe. We want to figure out a way to decrease that cost, break that down into specific biomarkers. And then, hopefully, when we move into our Phase Two of Remission Biome—which is actually bringing this work to the patient population—we might even be able to give people these tests, or at least provide them with very significant discounts for these tests.

    BL: Yeah, that’s something I wanted to ask you more about, too, is how you have been communicating with other people who want to try this sort of thing. I know from following y’all on Twitter and seeing some of the discussion around this project that folks are so interested. And generally, of course, there’s a big interest among people with Long COVID, ME, other similar conditions just trying to see what would work while there are no official FDA-approved treatments.

    How have you found that experience? And, as you look towards Phase Two, what are some of the things you’re going to be thinking about, in bringing these results to other people?

    TF: I’ve been talking to people on Twitter. And I would say, we’ve mostly been recommending, “wait until we do this first experiment, so that we can learn from it.” But for people who happen to be getting prescribed antibiotics [for an infection or something similar], then they have their doctor watching them. In that case, we can say, “you want to protect your gut, here are the probiotics that we’re doing, and you might possibly want to do a biome site test while working with your doctor.”

    TR: We have had such an overwhelming response. I think hundreds of people must have contacted you [Tess] personally now with a story that they thought might be a revision event, like the one that we’re describing. And hundreds more have said that they’ve had some sort of a positive or negative reaction to antibiotics. And they’ve had ME/CFS, or Long COVID. Those stories are really the jumping-off point for us. We want to put together a very formal survey to actually figure out how people are responding to antibiotics in post-viral illness, in general. There are a lot of clues in people’s stories.

    And the response of our community has been almost overwhelmingly positive, but there have been a few people who’ve had pretty severe baseline decreases after taking antibiotics. It’s always hard to know if the antibiotic itself was responsible for that decline or something else. But we really want to delve into that and figure out whether there’s a subset of people that seem to be having negative reactions.

    BL: I also wanted to ask about communicating the results from this work. What are you thinking about in terms of both sharing with other people in the community, and also, are you looking towards like a preprint, or scientific publication?

    TF: Yeah, we’ll be communicating in real-time with the community on Twitter, and getting input from all the scientists that we’ve talked to. That’s also something that we didn’t really mention yet is, we’ve talked to over 20 different researchers and gotten input from them. We have some that are interested in analyzing our results afterward. So, there’s community communication, but we are also planning to publish it, at least as a preprint.

    TR: We’re going to be a great case study. And it’s going to be a great paper. It’s not going to end up in a formal journal, but we’ll definitely pop it into one of the preprint servers.

    In addition to the case study, though, we really want to do a meta-analysis. A formal meta-analysis of all the studies that have ever looked at antibiotic use and post-viral illness. We think there are a lot of clues there. One of the things that’s really fascinating to us about this is that there are a number of similar situations that have happened in other conditions. One of the really key ones is PANS, or PANDAS, which often occurs in children when they get a staph infection—and then they get this crazy neurological event where their behavior changes and they develop OCD. Well, turns out, in an enormous number of those cases, if you give them antibiotics of the right type quickly enough—and a lot of the time, it’s amoxicillin plus minocycline, or doxycycline—you can actually completely put that child into remission.

    And there’s other disease groupings that seem to have these remission events, in very similar ways. Even Alzheimer’s, many people who have a grandparent with Alzheimer’s will tell stories of moments of complete lucidity. This indicates that maybe brain damage isn’t the ultimate issue, maybe there’s something going on with the communication networks. And that’s really what we’re targeting here [in our research].

    BL: That’s really interesting. I was actually just talking to another researcher for this story [Sonia Villapol at Houston Methodist Hospital], who mentioned that her lab, where she is studying Long COVID and the microbiome, has also done work on Alzheimer’s, and even traumatic brain injuries, where there’s some kind of microbiome interaction. I thought that was really interesting. It definitely seems like there’s so much more to be explored here.

    TF: Yeah, one of the researchers that we talked to is also doing ME/CFS and Long COVID research, and then Parkinson’s—using probiotics for Parkinson’s. I think a lot of what we’re doing can apply to other conditions, too.

    BL: Right. I also wanted to ask if y’all had any other comments about the two recent studies?

    TR: Well, I was pretty excited to see butyrate as sort of the highlight molecule. Very early on, we decided that we were going to try to increase our butyrate levels… So we were really excited to see that both of those papers linked to butyrate-producing bacteria, which was really key for us. It’s not just that we want to increase butyrate during the experiment itself, but it’s also a great way to actually help heal leaky gut issues. So it’s a really great intervention.

    BL: Yeah, that’s something that has come up a lot in the research I’ve done for this piece—the value of intervening early and trying to help people out before they’re going to have long-term symptoms, or at least in the earlier stages of illness. Which I know is one of the reasons why there’s so much interest in Long COVID, because you’re ideally diagnosing people earlier than what’s historically been the case for ME and some of these other related conditions.

    TR: To me, the really exciting application of this is that antibiotics are a very safe intervention that have been used for dozens of years. If it turns out that there’s a chance that taking a quick course of amoxicillin and a tetracycline like minocycline or doxycycline, can take someone who might have developed much more severe Long COVID and then MECFS out of that track, well, that’s phenomenal. If there was actually something that you could go to your doctor and say, “hey, I’m having these Long COVID symptoms, what can we do right now?” And the answer is, “well, it’s pretty safe, why not just give you a week’s worth of antibiotics?” It’s a pretty exciting possibility that we could stop some of these more severe cases.

    BL: Yeah, absolutely. I also wanted to ask, in terms of the institutional side of this, what do you think the NIH and other government agencies could be doing to better support this kind of work, and integrating ME/CFS and Long COVID research, as we try to understand the common mechanisms here?

    TR: Well, they could actually be treating us. This is really the biggest roadblock: there are hundreds of thousands of people out there who are undiagnosed and untreated, and are trying to biohack their way out of serious post-viral illness. None of these people are seeing clinicians that know what they’re talking about. So, we need treatment centers—but not just the Long COVID treatment centers, where they just tell people to rest, but centers that actually do biochemical testing, figure out what’s actually going on in their bodies, and then doing targeted treatment.

    Even if you can’t cure ME/CFS right now, it doesn’t mean that you can’t help people feel phenomenally better, and make sure that they don’t slide from moderate into severe. What you said before is so key, because most of us slide from moderate to severe when we get comorbidities. When we get MCAS on top of the Long COVID, for example. All of these comorbidities compound, and then they get people to a state where almost any intervention is aggravating to their systems, and they literally can’t tolerate light or sound or food. Treating someone at that stage is almost impossible, unfortunately, at this point. So early intervention is really key—but to get early intervention, you need clinicians that know what they’re doing interacting with patients.

    BL: Which we don’t have nearly enough of.

    TF: Yeah, that’s a major problem. Another angle that I’ll mention is more funding for research, specifically for ME/CFS. There are a lot of really great ME/CFS scientists who haven’t been able to get funding, but they’ve been trying to study this for decades. I keep hearing people saying that there’s these new researchers coming into Long COVID, who don’t really understand a lot about the history and what’s already been done. I think we need more funding for ME/CFS research, plus particularly funding for people who have been doing this for a long time.

    TR: Absolutely. One of the things that Tess and I are really excited about is—I guess what we’re going to be calling Phase Three, but it’s starting now—is we’re putting together a hybrid DAO, plus a nonprofit, to actually provide funding for researchers in this area. It’s not just for researchers, it’s specifically for PhD patients.

    We really want to tap into this community of sick scientists who’ve been sidelined by ME/CFS, by COVID, by other disabilities, and offer them the chance to actually get back into research in a way that they could do and would be supported. We’re really seeing a new model, a new way of being able to conduct research that is outside of academia, yet has checks and balances and support. Remission Biome has been, hands down, the best thing for my mental health that has happened in 10 years. If I could bring little bit of that to other people in my position—that’s what I would like.

    TF: We’ve actually had a lot of people volunteer to help. And I’ve gotten the comment many times, people saying, “this feels so good to use my expertise again.” These are people that have been on disability for 10 years and haven’t been able to do any work or any research. And now they’re able to give their expertise towards our project, and help us gain momentum and move forward, and they’re just really happy about it. I think there’s so much untapped expertise out there.

    BL: Yeah, that makes so much sense. And I hope I can keep following this project as y’all expand it.

    More on Long COVID

  • Q&A: Libraries lend out CO2 monitors to make public health data more accessible

    Q&A: Libraries lend out CO2 monitors to make public health data more accessible

    This map shows libraries across Canada that have set up CO2 monitor lending programs with CAVI.

    Two weeks ago, I shared that I’d recently purchased a monitor to measure CO2 as a proxy for ventilation in my apartment and other spaces. That post led to responses from several readers who’ve also been using CO2 monitors—including Kate Nyhan, a research and education librarian at Yale University who specializes in public health.

    Nyhan explained that, in addition to using a CO2 monitor at her home and workplace, she co-founded a nonprofit that helps public libraries loan out monitors. This nonprofit, called Community Access to Ventilation Information (CAVI), has brought CO2 monitor access to libraries serving about one in five Canadians. In addition to the monitor-lending, CAVI develops educational materials to help library patrons use these tools and collaborates with other air quality initiatives.

    I talked to Nyhan and Danielle Cane, CAVI’s managing director, to learn more about the organization and get their tips on using CO2 monitors. Here are the highlights from our conversation. 

    How CAVI started

    Cheryl White, an engineer and air quality expert based in Toronto, Canada, got the idea for this organization in fall 2021. At that time, many people on the COVID-conscious side of Twitter started to express interest in air monitoring, Cane said.

    “When we would post our readings from CO2 monitors on social media, a lot of people were saying, ‘This is really cool, I’d love to get involved,’” she recalled. “But it’s just so expensive to access these tools.” Higher-end monitors, like the Aranet device I purchased, can cost around $300. 

    White had the “bright idea” to partner with libraries as a way to make these monitors more accessible, Cane said. After Cane and Nyhan came onboard, the group reached out to Peterborough Public Library, a library system in a town northeast of Toronto. (Cane and White are both based in the area.)

    “Peterborough Public Library was really game to introduce this program,” Cane said. CAVI also worked with the local public health department, which helped gain buy-in for lending out air monitors. The initial Peterborough pilot was launched in spring 2022 with 15 monitors, supported by funding from Canadian and American Aranet distributors.

    The city of Toronto later joined the program, followed by other Canadian municipalities. Right now, about 22% of Canadians “have access to a co2 monitor through a public library,” Cane said. CAVI received additional funding in late 2022 to expand further. While the monitor-lending is focused on Canadian libraries, CAVI also produces free, open-source educational materials that can be used in other places.

    Why do this in libraries?

    Nyhan explained that libraries are well-poised to make air monitors more accessible. “Libraries want to empower community members with access to information,” she said. “In the context of indoor air quality, or COVID transmission risk mitigation, that might be information about airborne transmission, that might be about information about mitigations like air filters… It might be as tangible information as, what’s the number of CO2 parts per million in the space that I’m in right now?”

    Lending out CO2 monitors also fits into the “Library of Things,” a concept in which libraries lend out non-traditional items—ranging from home repair tools to arts and crafts materials. As an expensive item that can provide valuable information to the community, CO2 monitors are a great addition to many libraries’ existing collections. In addition, some libraries already have people on staff with public health expertise or existing programming in this area, Nyhan said.

    How the program works

    The lending system tends to vary from one library to the next, but most places are loaning out CO2 monitors for one week at a time, Cane said: “Especially given the demand in certain areas, like big cities, we find that the one week schedule tends to work out best to both balance, giving people the opportunity to check a variety of settings… and allowing other people to also have that same opportunity.”

    Some libraries have seen high demand, especially when the CO2 monitors first arrive at a new library. But as demand levels out over time, patrons might be able to “renew” their loans to keep the monitors for an additional week, Cane explained. Along with the physical monitors, libraries can point patrons to educational resources from CAVI that help interpret the findings.

    Interpreting CO2 monitor data

    As I’ve found in my own CO2 monitor adventures, there’s a lot these devices can tell you—but also a lot they can’t tell you. The biggest caveat, Cane explained, is that CO2 monitors are “a proxy for ventilation,” not a “proxy for infection risk.” In other words: a really high CO2 reading in a particular space doesn’t necessarily mean that infection is guaranteed, especially if other safety measures are in place. And “a really low reading doesn’t necessarily mean that you will not get infected,” Cane said.

    Aranet monitors have a built-in interpretation feature, marking certain CO2 readings as green (good), yellow (less good), and red (bad). But CAVI has produced materials that go into more detail about explaining the ppm (parts per million) measurements. Cane shared one document, designed in partnership with Toronto Public Library, which goes into detail on what higher CO2 readings mean and how to act on them.

    Taking action based on CO2 readings

    Nyhan used the example of a small car to explain how people may take simple actions based on their CO2 monitors’ results. “Even if it’s just a single person, because [a car is] such a small and tightly enclosed environment, you can very quickly see the impact of breathing out, or opening the window, or changing the air to recirculate or not,” she said. Opening a window or telling the car not to recirculate brings in more outdoor air, causing CO2 levels to get lower.

    This is a fairly simple lesson that a library patron might learn during the one week they have with a monitor, Nyhan said: “You learn that, and then you can give the CO2 monitor back to the library so someone else can use it.”

    In a larger space, actions based on high CO2 readings might include adding external air filters, opening several windows, or hiring an HVAC engineer to evaluate the ventilation system, Cane said. Not everyone might have the capacity to hire an HVAC engineer, but many people can buy or make air filters; Corsi-Rosenthal boxes are one popular DIY model that can be constructed with commercially available box fans and filters. CAVI has recently partnered with a Canadian high-schooler who’s worked to build these boxes and share accessible instructions.

    Nyhan also pointed out that CO2 monitors are “not just for people who control their own spaces” and can easily make ventilation changes. CO2 readings could also inform behavioral safety measures, she said, such as rapid testing before a social gathering in a poorly-ventilated apartment or avoiding certain poorly-ventilated parts of one’s workplace. In some cases, these readings could even be used to advocate for ventilation changes.

    Next steps for CAVI

    CAVI plans to continue expanding among public libraries in Canada. The organization also works with libraries elsewhere that may want to set up their own CO2 monitor lending programs, Nyhan said. Library staff and users are welcome to reach out to the CAVI team to learn more about the project: “We can share best practices, educational materials, assessment tools, grant proposals,” Nyhan said.

    While CAVI, like other air quality citizen science efforts, is currently focused on mitigating COVID-19 risks, its work has implications for many other public health threats. “If we have a wider acceptance of how respiratory pathogens are transmitted, it could be useful to help mitigate other viruses, other bacteria,” Cane said. Improving ventilation can lower the chances of infection for many pathogens and reduce the health risks associated with indoor air pollutants. 

    Lending out air monitors fits into work that some public libraries are already doing on environmental health, Nyhan said. Libraries might have existing programs about wildfire smoke, radon, and similar air quality threats; COVID-19 and pathogens like it provide motivation for expanding these efforts. “Indoor air quality, and environmental health more generally, is a hot topic that’s only going to get more important,” Nyhan said. 

    More on air quality

  • How Science Writers organizers planned the in-person conference’s COVID-19 safety measures

    How Science Writers organizers planned the in-person conference’s COVID-19 safety measures

    Masked Science Writers attendees watch a conference session. Photo by Betsy Ladyzhets.

    A couple of weeks ago, I wrote that the Science Writers conference—which I attended in-person—had great COVID-19 safety policies, better than other events I’ve gone to this year.

    The meeting of about 450 science journalists and communicators included required masks indoors, outdoor space for meals, and a vaccine requirement, among other safety measures. As I write this, about three weeks after the conference, there have been no reports of COVID-19 outbreaks (though the organizers were not requiring attendees to share all test results).

    I’ve previously reported on COVID-19 safety at large events, so I wanted to learn more about how the Science Writers organizers planned the conference and communicated policies to attendees. To find out, I talked to Tinsley Davis, executive director of the National Association of Science Writers.

    Davis shared behind-the-scenes insight into planning the 2022 Science Writers conference and tips for other event organizers. Her insight might be helpful whether you’re organizing a large journalism meeting or a small family gathering at Thanksgiving.

    Rather than transcribing the entire interview, I wrote a paraphrased summary (with quotes from Davis throughout). Let me know what you think about this format in comparison to past Q&As!

    Paying attention to COVID-19 news, planning in advance

    The Science Writers conference is a joint effort by two organizations, the National Association of Science Writers (or NASW, a membership organization) and the Council for the Advancement of Science Writing (or CASW, a nonprofit that runs awards programs, fellowships, and other initiatives). 

    Representatives from both organizations meet regularly on a steering committee to plan the annual meeting, Davis explained. In early 2021, that group paid attention to the vaccine rollout and started thinking about safety for a potential in-person meeting in the fall.

    “We are very fortunate to have, amongst our volunteer leadership, science journalists and science communicators, and people who are generally paying very close attention to not only what’s happening in the news, but what’s happening in the research community,” Davis said. “And they’re paying to attention to it both out of professional interests, but also out of personal interest… So we can leverage that in our discussions, and we don’t need to separately bring people up to speed.”

    These discussions led to initial plans for a hybrid meeting in October 2021, which would have both in-person and virtual components. The in-person conference planned to include a vaccine requirement and other safety measures. But in August, when the Delta variant surged across the country, NASW and CASW decided to shift the meeting to only virtual.

    Still, the 2021 plans and discussions proved to be helpful when the steering committee began to plan the next year’s meeting. “When we started the conversation in earnest in early 2022, we looked back to those policies that we had developed for 2021 and used those as a base to start thinking,” Davis said.

    Collaborating with the venue

    Science Writers 2022 took place in Memphis, Tennessee, with most events at the Renesant Convention Center. But unlike other conferences I’ve attended this year, most sessions with food were not held inside the convention center: the center had space available outside for people to eat and drink in a much lower-risk environment.

    I asked Davis about how she and the other organizers planned for outdoor dining. Staff at the convention center were very receptive to safety requests, she said: “We were met with such support and creativity.” This included closing down a street outside the conference center for one outdoor lunch event, and taking advantage of another outdoor area that was closed to traffic.

    It’s worth noting here, the weather really worked in favor of outdoor dining and socializing. For most of the weekend, temperatures were pleasant (in the 60s and 70s) and there was no rain. “It was a beautiful fall weekend,” Davis said.

    In addition to the outdoor plans, venue staff shared what they’d learned about COVID-19 safety from hosting other large conferences in spring and summer 2022, Davis told me. That included plans for how to arrange chairs in conference rooms for social distancing, and using security guards (required for crowd control) to help “gently remind” conference attendees to keep their masks on. Staff were also “readily able to talk about MERV ratings,” a measure of ventilation in the building, Davis added.

    Conferences like Science Writers typically aren’t able to extend their COVID-19 safety measures beyond attendees—in other words, the organizers can’t require convention center staff to mask up. But staff at the Resenant Convention Center “were very thoughtful about masking up” in conference spaces, Davis noted. (I observed and appreciated this as well.)

    Balancing safety and cost

    One of the best measures to reduce COVID-19 spread at a large event is rapid testing at the door, a safety policy backed up by scientific studies. But this type of mass testing can get pretty expensive for a conference of about 450 people, the size of Science Writers this year.

    “Cost is part of any decision,” Davis said. “And anytime you put the word ‘conference’ in front of something, it gets way more expensive. Like even a cup of yogurt gets more expensive when you have the words ‘conference catering’ in front of it.”

    Operating with a limited budget, the Science Writers organizing committee chose to prioritize an independent vaccine verification system, so that staff didn’t need to check all attendees’ vaccine cards upon arrival. They used the CrowdPass system and required everyone attending in-person to submit their vaccination information before traveling to Memphis.

    CrowdPass does offer on-site testing, Davis said. It would have been a great layer of safety, “but that was just an order of magnitude more expensive, and not something we could afford.” The conference also didn’t require attendees to report positive COVID-19 tests, though people were encouraged to stay home if they felt sick before the conference.

    Communication and control

    In the U.S.’s current COVID-19 environment, with rapidly-spreading variants and limited safety measures in most places, having an entirely COVID-free event is not really possible, Davis said. “What we did want to do is try and mitigate as much as possible in the spaces that we could control,” she explained.

    “Spaces of control” included vaccine verification before the event, required masks in the conference center, and prohibiting eating or drinking during sessions so that attendees stayed masked in those settings. Outside of the official event—in spaces like hotel elevators or nearby bars—the conference organizers had less control.

    But the safety policies for official events made it easier for attendees of varying COVID-19 risk comfort levels to participate. Making the conference broadly accessible was a priority for organizers, Davis said, as was providing safety information well in advance.

    “We tried to be very thorough, very clear, communicate early, and to really manage expectations,” she said. “We wanted someone to know, when they registered, exactly the kinds of precautions that we would be taking as a conference, and what would be expected of them as an attendee, and to really underlie it with the ‘why.’”

    Organizers aimed to clearly convey why this conference was taking COVID-19 safety so seriously: to help keep the community safe so that a wide group of people could participate. “We are so lucky as a community to be able to even ponder getting back together in person after two and a half horrible years, that we really owe it to each other, to be as thoughtful as possible,” Davis said.

    Overall, the communication strategy seemed effective: throughout the conference, Davis had to remind “exactly one person” to put a mask on, she said. It probably helped that many people attending the meeting had reported on COVID-19, or at least had closely followed pandemic news in their communities.

    “It was very heartening to see the level of, not just compliance with the COVID policies, but really the embracing of and the appreciation for them,” Davis said. Regardless of personal risk, everyone followed the conference policies.

    COVID-19 safety as a statement of values

    When I posted about the Science Writers conference’s COVID-19 safety policies on Twitter, one commenter pointed out that this meeting had “more precautions than some medical conferences.”

    I asked Davis for her thoughts on this comment, as well as how the safety measures on display at the conference showed NASW and CASW’s organizational values. “Not being a medical professional myself, but being someone who organizes conferences, I’m really proud of the values that we were able to bring forward and really proud of our community for complying with them, if not embracing them,” she said. She acknowledged, however, that as a relatively small conference, Science Writers might have been able to prioritize safety in a way that would’ve been more challenging for a bigger event. 

    Our conversation ended with a few other tips for organizers of large events:

    • Make COVID-19 safety “a continuing part of the conversation,” not just a “box that you need to tick off.” Organizers should keep an eye on the evolving COVID-19 landscape and be prepared to adjust their policies if needed.
    • Think about accessibility tradeoffs, such as when and for whom masks should be required. For example, Science Writers allowed speakers to take off their masks while at a distance from audience members so that people could read their lips and see facial signals if needed.
    • Take note of any tricky situations that come up and plan for the next year, so that safety measures and communications can continuously improve over time.
    • Put your COVID-19 policies online! Davis and other organizers found it helpful to look at public safety policies from other organizations. “Someone will find the experiences you’ve had helpful in craft crafting their own event, even if it’s much, much smaller than yours,” she said.

    More COVID-19 source spotlights

  • Unpacking U.S. data gaps and lack of public health action with Jason Salemi

    Unpacking U.S. data gaps and lack of public health action with Jason Salemi

    The CDC’s Community Level guidance contributes to current inaction on COVID-19 in the U.S. Image by Jason Salemi, from his June 10 Twitter thread.

    In April, the CDC launched a new center called the Center for Forecasting and Outbreak Analytics (or CFA). The new center aims to develop models of COVID-19 and other infectious diseases, while also helping public health agencies and individual Americans act on the information. One of CFA’s lead scientists compared it to the National Weather Service.

    But the problem is—as I discussed in a new story for FiveThirtyEight—the CFA currently does not have the data it needs to accomplish its goals. Among the challenges this new center is facing:

    • COVID-19 case data are becoming increasingly unreliable as PCR testing is less accessible and more people use at-home tests;
    • Hospitalization data are more reliable, but lag behind actual infections and may soon be unavailable in their current, comprehensive format;
    • Wastewater surveillance and other promising sources are not yet ready to replace clinical datasets;
    • A slow hiring process, as the center aims to bring on 100 scientists and communicators;
    • The CDC’s limited authority over state and local health agencies, and over the public.

    At the COVID-19 Data Dispatch today, I’m sharing one of the interviews I did for the FiveThirtyEight story. I talked to Jason Salemi, an epidemiologist at the University of South Florida College of Public Health, whom you may know from his excellent dashboard and Twitter threads providing detailed COVID-19 updates.

    While Salemi isn’t focused specifically on forecasting, he has a lot of insight about interpreting COVID-19 data and using the data for public health decisions. And I think he shares my frustration about the lack of safety measures that are being implemented across the U.S. at this dangerous point in the pandemic.

    For context, this interview took place about one month ago, while BA.2/BA.2.12.1 were driving a surge in the Northeast but hadn’t quite hit other parts of the country yet. This interview has been lightly edited and condensed for clarity.

    Betsy Ladyzhets: I wanted to start by asking, what do you see as the current state of trying to keep track of COVID in the United States? Like, what are some of the metrics that you’re looking at right now? What are some challenges that you’re facing as we deal with case numbers becoming less reliable?

    Jason Salemi: Definitely the case numbers issue. Throughout the entire pandemic, we all know that the case numbers that we learn about when somebody actually tests positive and that information gets recorded somewhere, reported to a State Department of Health and ultimately to the CDC, that’s always underestimated the true number of infections that’s been circulating in the population. Obviously, very early in the pandemic, that was really, really bad—we were mostly picking up people as they were getting sick and landing in the hospital. But as testing expanded, obviously, we did a much better job of being able to gauge what was happening with true infections by relying on the reported case numbers.

    However, during Omicron, and especially with the increased use of at-home testing—a lot of those at-home tests, if the person tested positive, were not making their way into a system that would actually get translated into the officially reported numbers. And negative at-home test results—those are definitely not making their way to public health agencies. I think in some jurisdictions, you were finding that 30%, 40% of all testing was actually antigen testing, and a significant portion of those were the at-home antigen test. 

    More recently, I think the official numbers that we hear about on a daily basis in terms of official COVID-19 cases, is becoming an increased undercount of the true number of infections that are circulating. Which is pretty striking, considering how much we’ve seen the numbers go up in the past few weeks. So, relying on officially reported cases does mean a lot less. But I still do believe that if you’re looking at—not necessarily where the numbers are exactly, but the trends in the numbers, how those numbers are changing over time—you can at least get a good feel for whether or not things are getting better or worse, even by using the COVID-19 case numbers. 

    Now, when you supplement that with things like wastewater numbers, data that are not biased by people taking advantage of testing or how they test, the wastewater numbers are maybe a better gauge for truer trends in the amount of viral spread. But again, even with wastewater numbers, two big things about those: number one is, it’s certainly not available, at least not that I can tell, for a lot of jurisdictions throughout the United States… It’s not available consistently across the country. 

    And number two, there’s nothing in those wastewater numbers where you actually can gauge: okay, this is the actual level of infection. What it helps us to do is, it’s a leading-edge indicator, where early on, we can say, “Oh, wow, we see an increase, a pretty pronounced increase in a particular area over time.” And hopefully, if we were doing things proactively, we could use those data to then implement some sort of concerted mitigation. So, this issue has become more of a challenge. But in many communities, we still can rely on how case numbers are changing over time to loosely gauge transmission rates. 

    Then, of course, a lot of people say, “It’s all about the hospitalization data, let’s utilize that.” Although I’d always love more metrics included in that [hospitalization] data set, it is something that, for some time now, we actually have consistently measured, at the national level, for every single state. You can get down to the hospital level, in some cases, and even by age group. We can have a decent understanding of how many people are being hospitalized with COVID 19. The nice thing about that is the consistency, and the fact that this [hospitalization dataset] is available everywhere, and we therefore have a decent resource that is capable of picking up indicators of more severe illness.

    But there are a lot of problems with the hospitalization data: namely, it’s a lagging indicator. Ultimately, if we were to rely exclusively on COVID-19 hospitalization rates and hospital capacity issues—those indicators lag new infections, often by five to seven days, at least. So, by the time we see those particular metrics rise, we will have lost valuable time to prevent morbidity and mortality. That’s the big [problem].

    The other thing is, there’s a lot of legitimacy to when people say, “Well, if a person went to the hospital for a non-COVID-19 related issue, and they just happened to test positive, they may not have been hospitalized because of COVID-19.” I think most are using the term “incidental.” Again, the numbers are not perfect. And when community transmission is as pronounced as it has been during many phases of Omicron, I think we do have a lot of situations where a lot of people are not being hospitalized because of COVID-19. But they are testing positive.

    For example, Jackson Health System in Florida was Tweeting out every day during the Omicron phase. And they would say—giving hypothetical numbers here—“We’ve got 250 people who are hospitalized, and that are positive for COVID-19. Of those 250 people, 51% were hospitalized for non-COVID-related reasons.” Some areas would give you more specifics, they would also break down by vaccinated versus unvaccinated. You get a lot of more rich, detailed data from some areas, but obviously, that’s not consistent across the country. In fact, I think it’s pretty rare.

    BL: Yeah, that point about hospitalizations being a lagging indicator is definitely something I want to highlight the story. And it seems very complicated, because I have heard from a couple of the modeling experts I’ve talked to that if you look at something like hospital admissions, specifically, that is less lagging. But still, overall, if you think about, like you were saying, trying to prevent more people from getting sick—even by the time you just see more hospital admissions, that’s still bad. You’ve still lost your chance to put in new mask measures, or whatever the case may be.

    JS: Oh, absolutely. And, you know, if we really were in a state right now, where getting infected really did no damage to people, it never caused any severe illness, we would obviously care less about transmission levels. Although you could always use the argument that the more we let COVID-19 circulate, the more likely it is that new variants will emerge with potentially more dangerous characteristics. So, even if it wasn’t causing a lot of severe illness, you’ve always got that aspect of it.

    But we are certainly not yet at a stage in which we can say [getting infected does no damage]—even though for the average individual Omicron is less severe when we compare it to something like Delta. But we paid a steep price in many areas in the United States to get the infection-acquired immunity and vaccine-acquired immunity that seems to be blunting the effects of Omicron. Right now, that’s why we’re not seeing the rise in hospitalization rates as steep as the rise in case rates. 

    But we are still seeing people getting hospitalized, an increasing number of people over the past couple of months. We’re not yet in a position where COVID-19 is not causing any damage. And we’re largely ignoring things like Long COVID. Just because somebody doesn’t get hospitalized, that doesn’t mean that [the virus is] still not causing a decrease in the quality of life for many people, and a decreased quality of life that can linger for some time.

    BL: Yeah, definitely. And then, another issue with hospitalization data that I wanted to ask you about, because I know you’ve looked at this, is the fact that if you’re using these county risk levels, or community levels, whatever the CDC is calling them—not every county has a hospital. So really, this is data at a somewhat larger regional level. I’m wondering if you could explain why this is an important distinction.

    JS: Yeah. And you know, this is not at all a criticism, this is kind-of the nature of the beast, so to speak. There are a lot of communities where—I’ll use Florida, because I’m most familiar with Florida, as an example. We have got a major health care system in Alachua County, which is really not a big county in Florida, not even in the top 20 largest counties. But it is a major area where a lot of people from surrounding smaller counties, like a nine- or ten-county catchment area, if they were to get really sick, that’s where they’re most likely going for treatment. And so, if you have a metric that is based on hospitalization rates, and you don’t have a hospital, obviously, you can no longer really provide a county-level indicator. It has to be more regional. And so you see a lot of variation in how the CDC has to now go from the county level to what they call health services areas.

    These [health service areas] are established groupings. In these regions, the overwhelming majority of people in these locations are going to a hospital in the broader health service area. And so it’s confusing, I think, to people: with this newer CDC metric, they wonder, “How is it that there’s no hospital in my county or the county next to me, yet you’re giving me a county-level risk measure that is supposed to be based primarily on hospitalization data?”

    And again, I think, some of the nuances of the metric get lost on people… Hospitalization data comes from a broader region [than cases], and there’s a lot of variation. There are some counties that are standalone, like Manatee County in Florida, so there is no health service area, it’s just one county for all measures. But there are some others where more than 15 counties that feed into that health service area. So again, for some people in some communities, I can understand where it’s just confusing and frustrating as to, “What does this risk level really mean, for me and the people that live near me, since the catchment region is so much larger?” This is not a right or a wrong, I understand why CDC does it the way that they do it if they’re trying to get a hospitalization-based measure. But it’s just challenging for people to digest.

    BL: Yeah, it’s challenging on that communications front. With the previous transmission levels, you could just kind-of look at the case rate and the positivity rate and be like, “Okay, I get where this is coming from.” But yeah, now it’s a little trickier. Another thing on this topic: I saw a report from POLITICO this morning that is suggesting, basically, if the National Public Health Emergency gets ended this summer, then the CDC might lose its ability to require states or hospitals to actually report the hospitalization data that is basically our best source right now. So, what would the implications be if that happens in a few months?

    (Editor’s note: After this interview, the Biden administration extended the public health emergency beyond July 15. But it’s unclear how many more times the emergency will be extended.)

    JS: I’d say pretty significant implications. Look, I’ve tried to give credit where credit is due, like the gains made with improving the federal hospitalization data. I’ve also been a critic when I feel as though we are missing key data sources or data elements. An example is the hospitalization data not having race and ethnicity information, I feel like that’s a big component that would be meaningful.

    But even with its limitations, the hospitalization data have been a very, very, very important tool for us to be able to report what’s happening in communities. And obviously, nobody wants to fly blind as it pertains to the pandemic. So if we don’t have uniform reporting from all of these states and jurisdictions, then we have to rely on the willingness of leaders at each state or community level to make similar information available, and to report that information in a timely and consistent manner. 

    Right now, we are fortunate that we continue to get the hospitalization data updated on a daily basis. And so yeah, that would obviously be a big loss if it were—it’s one thing to not have it required. But if states chose not to report that information, which certainly some states would choose not to… it would be a big loss, depending on what states choose to do to keep the population informed.

    Because, to be honest, when we get this national data, it’s a gut reaction that we want to compare states on everything—on death rates, on case rates, on hospitalization rates. To me, this can be a huge mistake. One of the obvious reasons that everybody talks about is age differences, right? Some states have a much higher percentage of older people. But it’s not just age that makes state comparisons difficult. It’s weather, and racial and ethnic distribution, and the job industries in which people can work, population density. So, I don’t really care too much about national-level data being used primarily to make state comparisons and inferences that can be misguided.

    But to have consistently reported information across the country, again, is important for us to be able to make more responsible decisions even at the local level. I would hope if that happens [losing the national dataset], we would still have states and cities and counties and communities and all these different geospatial areas continuing to report, collect, and make available to the public meaningful information in a timely manner so we can make responsible decisions.

    BL: Yeah, that makes sense. And I know that question of authority and like, what can and can’t you require the states to do, is a large issue for the CDC. I was able to talk to Mark Lipsitch yesterday, he’s one of the scientists who’s working on this new forecasting center. And one challenge he mentioned to me is that the CDC really doesn’t have the authority that it would like to in terms of requiring data reporting. They can’t require every state to start doing wastewater surveillance, they can’t require every state to report vaccine effectiveness data or breakthrough cases. And to me, that just seems like a massive hurdle that they face in trying to do this kind of long-term improvement of infectious disease forecasting.

    JS: Yeah—and it’s not just the ability, it’s also having the will. I’ve collaborated with some truly amazing scientists from the CDC for a very long time on a myriad of different initiatives, and I have little doubt that they will compile a team of experts that can analyze meaningful metrics to generate what I imagine will be a wealth of data on where we’re going in the pandemic. But it’s not just about analytic proficiency. I did read on their [CFA’s] site that their stated goal is to enable timely, effective decision-making to improve outbreak response. But how are we going to utilize those data to make recommendations? What outcomes are they going to emphasize? What communities are we thinking about when we make those recommendations?

    A lot of people talk about the measures we use, and which ones are best, and how we collect the data, and the validity, and the sophistication of the approaches that we use to either nowcast or forecast into the future. But to me, it’s also the way in which we operationalize those measures for public health recommendations. That’s where a lot of the talk is now about the measures being utilized by CDC. So whether it’s their four-level community transmission measure, or that newer three-level measure that’s based mainly on hospitalization data—how we’re using that to make recommendations, it says something about what the agencies who establish those boundaries are willing to accept.

    For example, I was just looking at some data again, when I did that thread this morning. The highest level on the community transmission metric, that used to indicate 100 cases or more per 100,000 people over the most recent seven-day window. Right now, based on the data that I just ran, we’ve got 105 counties in the United States with a population of at least 250,000—not just small counties, but large ones—that have a low community level [the CDC’s more recent metrics], the lowest possible, but they have a transmission level that is higher than that 100 per 100,000 threshold.

    And more importantly, we’ve got 28 counties—again, with a population of 250,000 or more—that are classified as medium level. That is a level with no recommendations for mask-wearing in public indoor settings. And those 28 counties have a case rate that is more than triple the threshold for high transmission, that’s 300 per 100,000, over the past seven days. You expect that medium level to change to high in the not-to-distant future for many of these areas.

    So again, it’s one thing to collect the metrics and have skilled analysis. But what we do with those measures and that analysis, is just as meaningful. And what does it mean, if we have an area that has really pronounced transmission—and we know in the past, that pronounced transmission means that the virus is going to be exceedingly good at finding vulnerable populations—and we’re not having any meaningful population-based recommendations… 

    When I looked, some of these counties were at like 400 per 100,000 [cases in a week], four times the threshold for the high transmission level [under the old CDC guidance], and they’re still not at a level where we’re supporting or recommending mask-wearing in public indoor settings. That’s pretty shocking. And I think that’s why anecdotally, now, even in my area, I’m just hearing about more and more people daily, that are not able to come to work. A lot of people are getting infected. And you’re seeing that in the rising numbers.

    BL: Absolutely. I mean, isn’t the threshold for moving from low to medium under the new community levels 200 new cases per 100,000 [per week, regardless of hospitalization numbers]?

    JS: Yeah, right. So even if you had no rise in hospitalizations, you can have a progression to the medium level. But that is now twice what the highest transmission threshold used to be. And again, I’m looking at counties that are in that medium level that now have almost twice even that newer threshold.

    We’re not yet in a situation where COVID is not causing any severe illness whatsoever. We’re ignoring a lot of the ramifications of Long COVID, we’re ignoring the fact that, when community spread has gotten so pronounced, you tend to have the virus easily, efficiently finding the most vulnerable people in those communities and still inflicting damage.

    I just feel like we’re missing an opportunity. We’re not talking about shutdowns, we’re talking about simple measures that we can put in place and recommend to people to try and balance having, normal living with putting reasonable but important precautions in place. Because that ultimately will prevent a lot of morbidity and mortality. And I feel like that’s maybe the big missed opportunity right now.

    So, I’d be excited to see a new forecasting center come out of the CDC. They are very adept scientists. But it’s ultimately, what do we do? What do we do with the data that emerges out of that center? And what recommendations, simple recommendations, do we end up giving to the public based on those analyses?

    BL: I totally agree. One of the new center’s focuses is that they want to hire a bunch of science communicators to think about these things. But still, I guess I’m a little skeptical about how much they’re gonna really be able to have an impact here, when we’re already at such a polarized position in the pandemic.

    JS: Yeah, it’s not that any of this is easy. No matter what you do, you’re going to upset a whole lot of people nowadays. I speak strictly from a scientist’s perspective. And I really do get all sides of this equation, like the businesses and the very real toll that the pandemic has taken on people. And so it is, no matter what you do, there is a balance that you have to achieve.

    But when I start to see—again, I’m going more from what has transpired specifically in Florida. And a lot of the talk this time last year, after we had the availability of vaccines, things were looking great for Florida. Numbers were really low. And that was pretty much throughout the United States, we had the vaccines, though we still heard a lot about protecting the most vulnerable, the oldest in our communities. And even as the cases started to rise, during Delta, it was like, well, just protect the vulnerable.

    But again, when community transmission gets that pronounced, the virus will continue to find the most vulnerable. And it ended up inflicting by far the largest death toll in Florida than we’ve had at any point in the pandemic, after vaccines were available for a long period of time. And that included a significant percentage of people who were not seniors. So, it’s tough, but still, people’s livelihood and lives are on the line when we’re talking about COVID.

    More federal data

  • How one wastewater plant became a leading COVID-19 forecasting source

    How one wastewater plant became a leading COVID-19 forecasting source

    The Metro Plant in the Twin Cities, Minnesota metro area has been tracking COVID-19 in wastewater since 2020. Dashboard screenshot retrieved on April 24.

    This week, I had a new story published with FiveThirtyEight and the Documenting COVID-19 project about the data and implementation challenges of wastewater surveillance.

    COVID-19 levels in waste—or, from our poop—have become an increasingly popular data source in the last couple of months (in this newsletter and for many other reporters and commentators), as PCR testing sites close and at-home tests become the norm. Wastewater can provide us with early warnings of rising transmission, and it includes COVID-19 infections from people who can’t or don’t want to get a PCR test.

    But wastewater surveillance is very uneven across the country, as I’ve noted before. A lot of local health agencies, research groups, and utility companies are now trying to expand their COVID-19 monitoring in wastewater, but they face a lot of barriers. My reporting suggests that we are many months (and a lot of federal investment) out from having a national wastewater surveillance system that can actually replace case data as a reliable source for COVID-19 trends and a driver for public health action.

    For this story, I surveyed 19 state and local health agencies, as well as scientists who work on wastewater sampling. Here are some major challenges that I heard from them (pulled from an old draft of the story):

    • Wastewater surveillance is highly sensitive to changes in a community’s coronavirus transmission levels, particularly when those levels are low, as has been the case across the U.S. in recent weeks.
    • Every wastewater collection site is different, with unique environmental and demographic factors – such as weather patterns or popularity with tourists – that must be accounted for.
    • While the CDC has led some coordinated efforts through the National Wastewater Surveillance System (NWSS), wastewater sampling techniques overall aren’t standardized across the country, leading to major differences in data quality.
    • Sparsely populated, rural communities are particularly challenging to monitor, as their small sizes lead to even more heightened sensitivity in wastewater.
    • Wastewater data is hard to communicate, especially when public health officials themselves aren’t sure how to use it. The CDC’s NWSS dashboard is a prime example.

    As bonus material in today’s COVID-19 Data Dispatch, I wanted to share one of the interviews I did for the story, which provides a good case study of the benefits and challenges of COVID-19 surveillance in wastewater.

    In this interview, I talked to Steve Balogh, a research scientist at the Metropolitan Council, a local agency in the Twin Cities, Minnesota metro area that manages the public water utility (along with public transportation and other services). Balogh and his colleagues started monitoring Twin Cities’ wastewater for COVID-19 in 2020, working with a research lab at the University of Minnesota. 

    Balogh gave me a detailed description of his team’s process for analyzing wastewater samples. Our conversation also touches on the learning curve that it takes to set up this surveillance, the differences between monitoring in urban and rural areas, and the dynamics at play when a wastewater plant suddenly becomes an important source for public health information. Later in the interview, Bonnie Kollodge, public relations manager at the Metropolitan Council, chimed in later to discuss the wastewater data’s media reception.

    This interview has been lightly edited and condensed for clarity. Also, it’s worth noting that the interview was conducted in early April; since then, COVID-19 levels have started rising again in the Twin Cities metro area’s wastewater.

    Betsy Ladyzhets: The first thing I wanted to ask about was, the backstory of sampling at the Metro Plant. I saw the dashboard goes back to November 2020, and I was wondering if that’s when you got started, and how that happened?

    Steve Balogh: We actually started looking into it in April of 2020. And we contracted with Biobot at that time… But in May, their price went up, so we started looking for alternatives. Then, we started a partnership with people at the University of Minnesota Genomics Center, who know about measuring RNA in things.

    At that point, we tried to figure out how to extract the RNA from our samples. They [University of Minnesota researchers] didn’t know anything about wastewater, but they knew everything about RNA. We know all about wastewater, but we don’t know anything about RNA. So it was a good match.

    That summer, [the university researchers] started trying to do the extractions and it didn’t really work out so well… So we said, “Okay, we’re going to try this.” By September of 2020, we had built our own lab, and we were trying out our own extractions, based on what we were seeing in the literature, and all the preprints that were piling up. In October, basically we settled on [a sampling process] that worked. And by November 1, we were actually getting data.

    BL: Yeah, that definitely aligns with what I’ve heard from some of the other scientists I’ve talked to who have worked on this, where it’s like, everybody was figuring [wastewater sampling methods] out on their own back in 2020.

    SB: Yeah, it was on the fly. Papers were coming out daily, just about, with new ideas on how to do things. And we had, like, four different extraction methods that we wanted to look at, also looking at sludge, in addition to influent wastewater… Honestly, it was pretty much pure luck that we settled on [a method] that really, really worked.

    We tried to get daily samples, and to put up numbers and see what [the data] looked like. And it actually did work—it actually tracked the reported caseload quite well. We figured, well, it must be working. We also did QA [quality assurance] in the lab, spiking the samples with known amounts of RNA, and trying to get that back. And all of that came back really well, too. So, we have a lot of confidence in our method.

    BL: So that [QA] is like, you put in certain RNA, and then you check to make sure that it shows up in the sample?

    SB: Exactly, yeah.

    BL: What is your process for analyzing the samples and distinguishing those trends, like seeing how they match the case numbers?

    SB: We do the extractions at our lab, with the samples from the Metro Plant. We take three milliliters of wastewater and we add 1.5 milliliters of something called Zymo DNA/RNA Shield, from a company in California called Zymo. That’s a buffer that stabilizes the RNA—it basically explodes whatever virus particles are in there, breaks them up, and then it stabilizes the RNA in the sample. So you can actually store those samples at room temperature for days, or maybe even weeks, because the RNA is stabilized.

    Then, we put that treated sample through a two-step extraction process. The first step is, we put the whole thing into a Zymo III-P column, combined with 12 milliliters of pure ethanol, and run that through the column. This is a silica column, so the RNA in the sample binds to the silica. Then we wash it and elute that RNA in 200 microliters of water. And then we take that 200 microliters, and run it though the second stage, which is just a smaller silica column. The RNA that’s in that 200 microliters binds onto the smaller column, and then we wash it and elute that into 20 microliters.

    Our total concentration is going from three milliliters of wastewater down to 20 microliters of pure water. That’s a concentration factor of 150. We figured that would work for pretty much most situations, and it’s turned out to be true.

    Then, we store those samples at minus 80 degrees Celsius. Until we take them over weekly to the University of Minnesota, where they do droplet digital PCR, RT-PCR, to amplify and detect the RNA that’s in our samples. We started out just getting the total viral load back in November 2020. But then, in the early part of 2021 when Alpha showed up, we started doing variant analysis as well. We’re now also looking for specific mutations that distinguish the different variants of concern, like Alpha and Delta and Omicron.

    BL: So, you take the samples every day, but then you bring them over [to the university] once a week, is that correct?

    SB: That’s correct.

    BL: When you’re getting that data, coming from the U of MN lab, what are you doing to interpret it? Or, in communicating the data on your dashboard, what are the considerations there?

    SB: We work up the numbers and calculate a total load of the virus, or the particular variants, that’s coming into the plant. And then we basically put that up on the dashboard. There’s not a whole lot of interpretation or manipulation of the data—we’re simply importing the load, basically, of what we see coming into the plant. The load is the concentration that we’ve measured in the sample, times the total volume of wastewater coming into the plant. 

    We think that’s a sufficient normalization procedure for a large wastewater treatment plant. I know some groups are using other normalization techniques, but we think load is sufficient to tell us what’s happening out in our sewer shed.

    BL: Yeah, that makes sense. I know this gets more complicated when you have smaller sites, but your sewer shed is serving a big population—

    SB: Almost two million people. Yeah, it’s a big sewershed. If you had 50% of your population leaving during the day to go to work in the next community, that would be something that you might have to consider using other normalization techniques. But that just isn’t the case [in the Twin Cities]. We see a pretty steady signal here.

    BL: Makes sense. Have you considered expanding to other sites? Or are is the plan to just stick with sampling at the main sewer ship location?

    SB: We already have, actually. We operate nine different wastewater treatment plants in the seven-county metro area. And we’ve already expanded to three of those other sites, so we now have four total plants that we’re taking samples at and having them analyzed at the Genomic Center. It only started within the last month, so we don’t have quite the database to really start showing it on our dashboard yet. But when we do [have more data], our plan is to put that up [on the dashboard] as well.

    BL: Do you have a sense of how much time it might take before you feel the data is useful enough to put on the dashboard?

    SB: Part of the problem has been, all of these samples that we’re getting from these other plants, we’re just taking the entire sample over to the Genomics Center, and they’re doing the extractions. They’re using my extraction procedure, but they’re doing it in their lab. So, there was some learning curve for them to figure that out. And also to hire staff and come up to speed in terms of facility, and procedure, and people… Now, it’s been a few weeks, and I think they’re just about there [in getting a handle on the RNA extraction methods]. So, I think our data will start to shape up pretty quickly. 

    Another thing that may be keeping us, at this point, from showing the data is, nothing’s happening. We’re at this bottom [with low coronavirus levels in the wastewater] where everything just looks noisy, because nothing’s changing. But as soon as we start to go up, and if we get higher—the current position is just going to look like a flat line. But right now, people could look at it and say, “Well, that’s just junk.”

    So, in that sense, we just don’t want to confuse matters and say, “Here’s a bunch of junk for you to look at. We want to put it into some context. And the context really is, when things start taking off, then you see, “Oh, it used to be very low. And now it’s very high.”

    (Editor’s note: Since this interview was conducted in early April, COVID-19 levels have started rising in the Twin Cities metro area.)

    BL: That makes a lot of sense. Also, I hear you on the challenges of learning these methods. I was a biology major in school, and I worked in a lab, briefly, that did RNA extraction. And I remember how tricky it is, so I can envision the learning curve.

    SB: Well, these are experts at the Genomics Center, they know what they’re doing. But I think even they have been surprised at how how robust the viral RNA is in wastewater. A lot of people at the beginning of this pandemic said, “You’ll never see it in wastewater. It’s RNA, RNA is very sensitive, it’ll break down.” But that just isn’t the case—the RNA is quite robust in wastewater, and the signal lasts for a long time. It has to last for many hours, for it to travel from the far end of our sewershed to get to us [at the treatment plant]. And then, even in the refrigerator, when you refrigerate just the raw sample, it’ll stay in a reasonable concentration without dropping too much for days.

    BL: What has the reception to this work been from the public, the state health department, or from local media or other people who are using and watching the data?

    SB: It’s been incredible. You can ask Bonnie more about it.

    Bonnie Kollodge: It’s ginormous. I mean, it just has spread everywhere. I don’t even know the social spread, but I think somebody was tracking our impressions in print and online media… I think there were, like, 11 million impressions between January and the end of March. And we get lots of requests for Steve’s time, lots of requests for a daily accounting [of the data]. 

    When we began this work, it really was out of public service—seeing that there’s a pandemic going on, and what can we do to help? That’s when they started developing this idea, then working with the Depratment of Health, which is really our state lead on this [COVID-19 response]. They came to rely heavily on our information, to compare it against what their test results were showing. Then, as people started to do home testing, that was a whole other factor. It was really wastewater that was taking the lead on showing what was happening with the virus and the variants… 

    Every week, we put an update online, and reporters go right to it, to determine how they’re gonna position [their COVID-19 updates]. Steve also provides, in addition to the data, a little narrative about what’s happening that helps reporters—some who are very conversant in data, but others who are not—it helps them it understand what we’re seeing.

    BL: I can see how that would be helpful, especially if you’re releasing a week’s worth of data points at once. You sort-of have a mini trend to talk about.

    BK: Yeah, and we send it to the governor’s office, and to the Health Department. They appreciate the transparency… They know what’s happening [with the virus], and can adapt.

    BL: Right. And Bonnie, you mentioned something I wanted to ask more about, which is how the increased use of at-home tests and lower availability of PCR tests has increased the demand for wastewater data in the last few months, in particular. Now that you maybe have less reliable case data to compare against, has the thinking and interpreting the wastewater data shifted at all?

    SB: I think we’ve actually had that statement from reporters. They’ve said, “We can’t trust the testing data anymore. And it’s going to be wastewater from here on out.”

    BK: Just this week, there was a reporter who asked to get early results tomorrow. And he said, “This [wastewater data] is what I’m watching.” … The public has glommed on to this resource as a demonstration of what’s happening. And, like Steve said, it’s not a small sample. There are almost two million people served by this by this particular plant.

    BL: From what I understand, part of what can be really helpful [with wastewater data] is when you have that longevity of data, as you all do. You have a year and a half of trends. And so when you see a new spike, it’s easier to compare to past numbers than for other parts of the country that are just starting their wastewater surveillance right now.

    SB: Yeah. I think the other thing that has been really useful for our [state] department of health is, they’ve really appreciated the variant data that we have. That was really the first thing that got their attention… And we were giving them [variant] data ahead of time. The clinical tests were taking days or weeks to come back, and we could give them variant data the same week. So, that was the first thing that got our department of health here interested. But when they saw that we can track trends, they recognized that this has value at lower levels when testing goes away, basically.

    BL: How would you want to see support from the federal government in expanding this wastewater work? Like you mentioned, getting it in more treatment plants, and any other resources that you feel would be helpful.

    SB: Well, I think that’s underway, as we understand it, with the National Wastewater Surveillance System, NWSS. I think they’re funded through 2025, and I think the goal there is to basically sign up as many treatment plants as they can in the country.

    (Editor’s note: This is accurate, per a CDC media briefing in February.)

    Hopefully, that’s the beginning of something that is going to go beyond the pandemic, and give us a measure of community health in the future. Because wastewater is a community urine test, basically. It’s everybody contributing, and it can be useful for other pathogens and viruses in the future. So, yeah, [expanding that network] would be great. Let’s do it.

    BL: Do you envision adding other viruses to the testing that you’re doing? Flu or RSV are ones that I’ve heard some folks are considering.

    SB: Yeah, that would be something to do going forward for us. Though, it’s not clear how long we continue this work, just because these other projects are expanding, like the national project. And even our department of health here [in Minnesota] is talking about bringing this type of analysis into their own laboratory. Certainly going forward, long-term, that would be a goal for any work done here in Minnesota—to add those things to the menu of what we’re analyzing.

    BL: Right. So you might be taking the samples to the Minnesota health department instead of the university, or something like that?

    SB: Someday. Yeah, we just don’t know at this point.

    BK: This is an evolving scinece. And this is not what we typically do—I mean, we do wastewater collection and treatment. So this [COVID-19 reporting] is a little outside of our regular parameters. But, like Steve and his superiors have been saying, this is an evolving science, so let’s see where this takes us, in terms of infectious disease.

    It’s funny, when I go out and talk to people and say, “I work for the Met Council, and I help in communications with the wastewater analysis,” everybody knows what I’m talking about. It’s just so much out there. But I think that these things [testing for other diseases] are all being explored, and this has really opened up new possibilities.

    SB: From the beginning, it’s just been a scramble. You don’t know what’s going to be coming. What I’m doing, a lot, is trying to get ourselves in a position so that, when the next variant of concern pops up, we have an assay that can measure it. There’s still a lot of unknowns about what’s going on, and everything’s new every day, just about.

    More state data

  • A new resource for journalists covering Long COVID

    A new resource for journalists covering Long COVID

    Screenshot of the source list, showing some of the main how-to info available.

    This week, a new resource that I’ve been working on for the past few months went live: a comprehensive source list including Long COVID patients and experts who are willing to talk to reporters. This source list project was a collaboration with Fiona Lowenstein, who’s a journalist, speaker, consultant, and founder of the Body Politic support group for Long COVID patients.

    Here’s some info about the source list:

    • It includes over 300 Long COVID sources from the U.S. and other countries, spanning all ages, race and ethnicity groups, and other demographics.
    • It’s sorted into four categories: patients who identify as Long COVID experts and/or advocates; patients who aren’t experts but can speak to their own lived experience; other experts (scientists, clinicians, advocates, etc.); and related conditions and experiences.
    • Patients and experts have identified topics about which they’d like to talk to journalists, including Long COVID research, patient care, policy, mental health, relationships, financial insecurity, and related conditions (such as ME/CFS and dysautonomia).
    • The list is hosted on Notion, allowing users to search and filter for specific source needs.
    • This project is ongoing, and we will be adding more sources on an ongoing basis. If you would like to be added or have other feedback, please email LongCovidSourceList@gmail.com!
    https://twitter.com/fi_lowenstein/status/1481399096711606273

    To further explain the motivations for this project and provide some advice on how to use the source list, I did a Q&A with Fiona. Our conversation included the gaps in Long COVID news coverage, connecting the dots between Long COVID and other chronic conditions, recommendations for interviewing Long COVID patients, suggestions for covering this condition in year three of the pandemic, and much more. This interview has been lightly edited and condensed for clarity.

    Betsy Ladyzhets: Why did you want to do this project? Why was it worth putting the time and effort into making this source list?

    Fiona Lowenstein: I think there were two things. One was almost like a personal desire to have fewer media inquiries in my own inbox. I was receiving a lot of emails from journalists who were looking for very specific types of Long COVID sources. Part of that was because I started the support group Body Politic, and people were reaching out, asking me to post stuff in the group. Also, I have written a lot of stories on Long COVID and interviewed a lot of patients, and so people wanted help reaching more patients.

    I knew that a lot of the support group leaders were very burnt out and kind-of exhausted, and that media requests are one of the biggest sources of, like, email stress. And I wanted to think about, is there a way to just ease this process for everyone? I was also noticing that journalists were getting frustrated with how long it was taking to get in touch with [Long COVID] sources, because so many of these groups are run by chronically ill people, and a lot of them are volunteers. They’re not always able to respond to an email in twelve hours.

    Part of [the motivation] was also feeling like the news coverage of Long COVID, a lot of it focuses on the same people and the same stories. I’m someone that has been included in a lot of those articles, and at a certain point in time, I stopped doing press on my own experience, because I was like, this story is already out there. And I’m not sure it’s even reflective of the average experience of Long COVID, just because I had a lot of privilege that helped me get care and rest through my recovery.

    So, I wanted to see more types of patients talked about, more patients who aren’t necessarily young and super healthy and fit before they got sick. Because that was very much the narrative for a long time. And that is sort-of an ableist narrative, to be emphasizing so strongly that so many of us were young and healthy, and we should care about our chronic illnesses because of that.

    Also, I know that Long COVID coverage is going to have to go deeper and is already starting to go deeper in the coming year. Most news outlets have had at least one story explaining what Long COVID is. But we’re now at a point where we’re going to have to delve deeper into, like, what are the financial risks? What toll does [Long COVID] take on relationships? How are people navigating workplace accommodations? What about these specific symptom clusters that might morph over time? What about people who have additional diagnoses [of other chronic conditions] on top of having Long COVID?

    And the last thing was, I want to connect the dots between Long COVID and other post-viral or infection-initiated chronic illnesses, like ME/CFS, dysautonomia, and other diagnoses that people with Long COVID have received. These are also diseases that have a lot in common, both in terms of symptoms and the way that they present but also in terms of social and political issues with regards to getting care, getting funded research, etc.

    I think those of us who have had Long COVID and been involved or even been a fly on the wall in this advocacy work have seen how people with related chronic illnesses are not getting as much media attention. Even though they are really helping the Long COVID advocacy movement in a huge way, and helping patients on a day-to-day basis. So, that was why I wanted to include people with related conditions and experiences [on the source list], ideally, as well.

    BL: Yeah, that makes sense. When we were starting to put together the Google forms [used to collect source’s information] and thinking about who we wanted to send them to, what were some of the things that you were considering?

    FL: I was thinking a lot about the patient side of things. I wanted to connect with the leaders of the big [Long COVID] support groups, especially the private support groups, because the private support groups are a little more insular and more highly moderated. They’re the places where we can assume that a larger majority of the members actually have Long COVID. But the private support groups also have no way for journalists to kind-of see into those ecosystems. So, I wanted to connect with those support group leaders and have them share [the project within their groups]. I also did a lot of sharing on social media, because I’m followed by a lot of Long COVID patients and people living with the illness. And I messaged past sources, other people that I’ve talked to. 

    That being said, I was a little surprised—we got a lot of patient responses, but I think we could have gotten more. (Editor’s note: The list includes over 250 patients and 80 non-patient experts.) I know that there are more [Long COVID] people out there who want to tell their stories. But I think that, among the population of people with Long COVID who want to talk to the press, there are a lot of people who are just burnt out and tired of filling out forms. And there’s also a lot of distrust of the media. There was at least one support group that basically said, “We don’t want to participate in this because we don’t want our members’ information out there for reporters to access, we’ve had so many bad experiences with journalists.”

    That was a tricky thing to navigate. To that end, something I’m hoping to do at some point is organize another media training with some of these support groups, to talk through, like, what are some of the issues that are coming up in the journalist-patient relationship? What are your rights as a source when you’re being interviewed? That sort of thing.

    BL: Was there anything else that surprised you, when you were looking at the form responses? I know one thing that struck me when I was looking at them was, how many patients checked the box for experience with financial instability. I knew that was an issue, but it’s not something that a lot of articles have focused on so far.

    FL: Yeah, that’s a really good point. I filled out the form, because I’ve had Long COVID, and I checked that off, too. I think that, in the Long COVID communities, [financial instability] is such a huge issue, and it’s being talked about constantly. Even for people like myself—I had a relative amount of financial privilege, I had savings that I could rely on after I got sick and couldn’t work. And I had my parents and my partner’s parents, they were able to contribute a little bit to our rent and our groceries and that sort of thing. But it’s still massively set back my finances. So, I think almost everyone has had that experience on some level.

    I think one thing that surprised me was how many people had a diagnosis of another condition on top of Long COVID. It’s good, it’s heartening to see that people are getting diagnosed with things like dysautonomia, myalgic encephalomyelitis, and mast cell activation syndrome. I know that [these diagnoses are] happening a lot with the patient advocates that I know, but those are people who often have the highest levels of access to a clinician or a Long COVID clinic. But [the form responses] made me wonder if maybe also, there’s been so much information-sharing online and between patient groups that people are now able to diagnose themselves with this stuff—which is very common in chronic health communities because it can be so hard to get a diagnosis. 

    So, it was interesting seeing that so many people have checked off dysautonomia and these other things, because it made me feel like, okay, there actually is a pretty large group of people that are very aware of these other illnesses. I could not have told you what dysautonomia was, prior to getting COVID—even though I technically had a form of it, it turns out, before getting COVID. It seems indicative of the extent to which community information sharing has spread, and actually helps people figure out what they’re dealing with.

    And those diagnoses are also really helpful for figuring out your symptom management techniques. Like, I learned rest and pacing from people in the ME community. So that’s a huge part of it, too: it’s having that community that you can look to, in addition to Long COVID. People who have been sicker for longer, and research has been going on for longer, and you can learn from [these other chronic illness patients].

    BL: What recommendations do you have for people who are using the source list?

    FL: There are a couple of kind-of broad stroke recommendations that we tried to account for in some of the questions we included on the list.

    For example, one thing that I have said to colleagues and also publicly throughout the pandemic is: if you are a journalist reporting on Long COVID, you unfortunately have to have a pretty flexible schedule with your interviews, because this is an unpredictable illness. Someone could tell you, “Yeah, I’m available tomorrow at 2pm.” And then they could end up being really sick at that time. So, in the questions for patients, there’s a space where they can indicate how quickly they think, on average, they’ll respond to reporters. Hopefully that will help with this issue of, the urgency of tight deadlines, while trying to report on an issue in which people can’t always get back to you in a short timeframe.

    Just be aware that these are people who, even though a lot of them may not be working, are dealing with a full schedule of managing their own health. It’s also important to know that there is distrust amongst this population, in terms of interacting with journalists and reporters. I’m not saying that exists with every single one COVID patient, and lots of people have had really good experiences talking to journalists.

    But still, for that reason, it’s sometimes helpful, when you’re interviewing someone with Long COVID, to explain why you’re asking the questions you’re asking. For example, on the source list, we’ve included both people who have tested positive for COVID-19—via a PCR test or other diagnostic test—and people who have not tested positive. Some of those people who have not tested positive have a clinical diagnosis of COVID or Long COVID, while others do not. It’s important to understand the difference between those testing statuses and those diagnosis statuses. These statuses may have something to do with how intense the person’s acute symptoms were, but it has more to do with where the person was at the time that they sought the test, what time in the pandemic it was, and what sorts of privilege they might have or not have within the healthcare system in terms of accessing a test. Like, do they have a car and can they drive themselves to get a test? 

    That [testing status] question comes up a lot. And I think that journalists, when asking about just testing status, a helpful thing to do is explain why you’re asking. You could say, “I’m going to include people of all testing statuses in this article, because I understand that not everyone who has Long COVID was able to get a positive test for an acute COVID infection. But just for the purposes of accuracy, I need to ask, are you someone who had a positive COVID test or did you not have a COVID test? Because I need to include those details.” 

    Also, some people on the list have specified different methods of interview that they are comfortable with. But it never hurts to ask and double check [about interview method]. There are people who have really intense screen sensitivity and light sensitivity, and so emailing is going to be more difficult for them. Then, there are other people for whom a Zoom call or phone call is actually going to be more difficult, and they’d prefer texting, or emailing, or audio messages. I know a lot of times there is sort-of a reticence with journalists to use methods other than a Zoom call. But a lot of Long COVID patients have been communicating super effectively using nontraditional means for the entire pandemic. So, have a bit of trust in their ability to do that.

    BL: Are there any other things that you want journalists to know about talking to Long COVID patients?

    FL: There are a lot of things! One other thing to keep in mind is that everybody has a different level of expertise on Long COVID. A lot of times, I’ll seen articles—or I’ve even been in this position—there are people on our list who have identified themselves as experts on Long COVID, or patient advocates, outside of just their lived experience. Those are people who can, yeah, they might be able to speak to their lived experience, but they’re also going to be able to speak to, like, what they’re seeing in their support group or their experience, trying to advocate for policy change.

    And I think it’s a shame when the stories about those advocates focus only on their own health issues. That happens a lot, just because I think journalists have a hard time finding people who are willing to talk about their own health issues. But be aware that there are a lot of people who have really a higher level of expertise than just, “This is what happened to me in my body,” people who have taken a lot of care and a lot of time to read the latest research on Long COVID and are in touch with doctors and scientists and policymakers. I think treating those people as experts on the subject is important.

    On the flip side, not every Long COVID patient is going to be able to speak to those macro issues, and not every Long COVID patient has the same understanding of what’s happening in their body. There are also a ton of people—and these people probably are not on our list, unfortunately—people who have Long COVID and don’t know what it is, or don’t know that it’s called Long COVID. So, knowing that people’s understandings will vary from individual to individual is important as well, I think.

    And don’t be afraid to interview people with related illnesses! Again, dysautonomia is an incredibly common diagnosis, it seems, for people with Long COVID. But I haven’t really seen many articles that are interviewing other people who have lived with dysautonomia for ten years, or scientists who are studying it, or that sort of thing. 

    BL: Yeah. How would you like to see the source get used?

    FL: We had a media inquiry today [via LongCovidSourceList@gmail.com] that excited me because it was about a really hyper-specific local story. I would love to see more of that. Because I think Long COVID is an illness that does radicalize a lot of people, through getting sick and seeing like, “Oh, no one’s there to take care of me, and the government doesn’t care.” And a lot of people who are angry are organizing in their communities, or they’re even advocating on behalf of themselves.

    I think, in the coming decade, we’re going to learn about more and more of these people who have been doing this [organizing] on the local level. Like, I know of many people who have organized really small support groups in their town or in their city. So, I’d love to see more hyperlocal coverage of how Long COVID is impacting individual communities. I’d also love to see more coverage of caregivers and people with loved ones who have Long COVID, and how folks are navigating those relationships. Because I think there are so many lessons we can learn about disability and chronic illness and relationships in general from those stories. 

    And obviously, I’d love to see more diverse sources. Near the beginning of the pandemic, there was a period of time where there were a lot of stories about health disparities. And we were talking a lot more about the impact of COVID on undocumented populations, or Black and Latinx and Indigenous populations in the US, or the people in rural areas or inner city areas.

    We haven’t really seen a lot of that coverage for Long COVID. Part of that is because no one’s tracking it on a nationwide level, like we don’t have the case counts for Long COVID that we have for COVID infections. But there’s still stories to be done [on this issue]. You can go into a community and all you really need is one person, one source that has Long COVID in that community, to understand: How is that community grappling with this condition? Does that person know anybody else who knows what Long COVID is? Is that person educating everyone in their community on what Long COVID is? How did that person find care? Is there a hospital near them? Those are the stories I’d really like to see more. 

    Those stories, with an emphasis on those populations that were hit hardest at the beginning of the pandemic, and are still hard-hit. Like, I saw the other day that in Los Angeles, where I am, homeless populations have some of the highest rates of COVID infections. That would be a really interesting story and a really important story to look at, what does long-term care look like for those populations? 

    BL: You mentioned the fact that Long COVID cases are not tracked the way so many other COVID metrics are—which, as a data journalist, I definitely consider to be one of the biggest data gaps of the pandemic. Are there any other stories that you would want to see in that vein, or any other coverage areas you would like to see around Long COVID?

    FL: Yeah. I think this idea would take kind of the right type of journalist, probably someone with a deep knowledge of chronic illness communities; it would be interesting to delve into what I was talking about before, in terms of these additional diagnoses that [Long COVID patients] have received and/or self-diagnosed themselves with.

    I’m also very curious about how people get diagnosed with Long COVID, because it seems to be happening in a different way with everyone. There are the people like me, where I don’t have a clinical diagnosis of Long COVID—I don’t think I do, maybe my doctor put something on my chart—but like, I just have a COVID PCR test, and then I have records of going for care for these other related problems. And then I have an additional diagnosis of this kind-of dysautonomia-related thing.

    We do now have an ICD code for Long COVID. But I’m not exactly sure that it’s being used in all situations. And like, if someone gets diagnosed with ME/CFS, are they getting diagnosed with both those ICD codes, or does one diagnosis overrule the other one? So, I think there’s a lot of interesting stuff there. You could also delve into how common it is to self-diagnose and what that looks like when you end up going to the doctor at some point later on. This [story] can be done in a variety of ways. People could also write guides on, “How do you get an accurate diagnosis?” And, “What does a diagnosis mean in terms of your insurance coverage, or your eligibility for disability benefits?”

    I think [disability benefits are] another thing that is going to be huge. I get a lot of emails from people with Long COVID who have been denied long-term disability. One person I was speaking to was from a Republican state, and she was saying, like, “I’m from a state where the government shut down all the COVID-related social programs earlier than in other states. Why would I believe the caseworker in my state is going to take my Long COVID disability case seriously?”

    I think that’s an interesting thing, too. COVID was highly politicized. Long COVID isn’t highly political in the exact same way simply because it’s not something that everybody knows about and is talking about. But there’s that question: if we know that COVID infections and COVID care can sometimes differ state to state based on the political leadership and what sort of funding has been put into healthcare systems, how does that look for Long COVID? What does that mean for people applying for disability benefits? Are people in blue states having an easier time getting approved for long-term disability? Does it not matter? That sort of thing.

    BL: Yeah, that’s a good point. Those were all of my questions—is there anything else that you think is important for people to know about this project or about using the list? 

    FL: Well, I’m curious—I know that you were writing about Long COVID and looking for sources, but I’m kind of curious why you wanted to participate in this project and why it felt important to you?

    BL: I think that, as I said a bit ago, I feel like this is a really important data gap. There’s this feedback loop where, we don’t have really solid numbers about Long COVID, and so people don’t know about it, and so that contributes to the lack of numbers, and then it sort-of spirals in that way. And this [project] seemed like a way to combat that situation, at least a little bit. And also, I like making resources for other journalists, it’s part of the reason why [the COVID-19 Data Dispatch] exists, basically. This project felt like an extension of that goal.

    FL: Yeah, that makes a lot of sense. I think there is an enormous desire for those of us who followed Long COVID from the beginning to see it get its due in the media. And I think, now that we’re entering year three, and we’re hearing that Omicron is potentially more mild—it just feels like, when is it going to be the time that we fully turn our attention to this? In terms of gathering the data and, and writing about it.

  • New CDC mortality data: “Real-time public health surveillance at a highly granular level”

    New CDC mortality data: “Real-time public health surveillance at a highly granular level”

    The CDC’s new data release allows researchers to search through mortality data from 2020 and 2021 in great detail. Screenshot of the CDC’s search tool retrieved December 12.

    This past Monday, the CDC put out a major data release: mortality data for 2020 and 2021, encompassing the pandemic’s impact on deaths from all causes in the U.S.

    The new data allow researchers and reporters to investigate excess deaths, a measure of the pandemic’s true toll—comparing the number of deaths that occurred in a particular region, during a particular year, to deaths that would’ve been expected had COVID-19 not occurred. At the same time, the new data allow for investigations into COVID-19 disparities and increased deaths of non-COVID causes during the pandemic.

    To give you a sense of the scale here: As of Saturday, the U.S. has reported almost 800,000 COVID-19 deaths. But experts say the true COVID-19 death toll may be 20% higher, meaning that one million Americans have died from the virus. And that’s not counting deaths tied to isolation, drug overdoses, missed healthcare, and other pandemic-related causes.

    The CDC’s new data release is unique because, in a typical year, the CDC reports mortality data with a huge lag. Deaths from 2019 were reported in early 2021, for example. But now, the CDC has adapted its reporting system to provide the same level of detail that we’d typically get with that huge lag—now with a lag of just a few weeks. The CDC has also improved its WONDER query system, allowing researchers to search the data with more detail than before.

    “I would describe this new release as more real-time surveillance at more specific detail than any journalists, or epidemiologists, or any other kind of researcher even knows what to do with,” said Dillon Bergin, an investigative reporter and my colleague at the Documenting COVID-19 project, at the Brown Institute for Media Innovation and MuckRock.

    Along with Dillon and other Documenting COVID-19 reporters, I worked on a story explaining why these CDC data are such a big deal—along with what we’re seeing in the numbers so far. The story was published this week at USA Today and at MuckRock. Our team also compiled a data repository with state-level information from the new CDC release, combined with death data from 2019 and excess deaths.

    If you’re a reporter who’d like to learn more about the new CDC data, you can sign up for a webinar with the Documenting COVID-19 team—taking place next Wednesday, December 15, at 12 PM Eastern time. It’s free and will go for about an hour, with lots of time for questions. Sign up here!

    Editor’s note, December 27: This webinar was recorded; you can watch the recording here.

    Also, as our initial story is part of a larger investigation (in collaboration with USA Today), the team has put together a callout form for people to share their stories around COVID-19 deaths in their communities. If you have a story to share, you can fill out the form here.

    To provide some more information on why this new CDC release is so exciting—and what you can do with the data—I asked Dillon a few questions about it. As the lead reporter on our team’s excess deaths investigation, he’s spent more time with these data than anyone else. This interview has been lightly edited and condensed for clarity.

    Betsy Ladyzhets: How would you summarize this new release? What is it?

    Dillon Bergin: I would describe this new release as more real-time surveillance at more specific detail than any journalists, or epidemiologists, or any other kind of researcher even knows what to do with. It’s unfathomably detailed, and the fact that we’re going to be able to see updates in almost real time is really critical at this stage of the pandemic, or at any stage in a public health crisis. I think it’s a huge, huge step forward.

    BL: Specifically in the realm of COVID deaths, but also, all deaths during the pandemic.

    DB: Exactly, yes. In the realm of COVID deaths, we do know that there is a large gap between the total amount of excess deaths and the excess deaths that COVID accounts for. So it’s interesting from that angle, understanding what COVID might have been misclassified. But the data can also be used for a broad range of other types of deaths that have happened during the pandemic or possibly increased during the pandemic.

    BL: So why are researchers excited about this data release?

    DB: Previously, for something to go up on the WONDER website, or to become WONDER data, has to be finalized in the year after. So, data from 2020 would just be finalized now. Typically, we might not see that data until, probably, early in the new year [2022].

    But with the new tool, we’re getting that 2020 and 2021 WONDER data now. And the CDC does a great job of providing a lot of granular details about causes of death, and racial demographics… Those are things that general CDC [mortality] data gives you, but the WONDER data is even more detailed. So, the fact that researchers don’t have to wait anymore for that data to be finalized, that the CDC is providing provisional data at such a detailed level—that’s what researchers are excited about.

    BL: It’s the provisional data that’s being released, like, a year earlier than you would normally expect it to be published, right?

    DB: Yeah, a year earlier than you would expect it to be published. Which means it’s almost real-time, because it has, I think, a three- or four- week lag. This data is real-time public health surveillance at a highly granular level—which is what people have been asking for. It’s what epidemiologists have been asking for, researchers, advocates of all kinds, journalists, lots of people have been saying, “We need this type of surveillance.”

    BL: When you say a three- or four-week lag—the CDC is going to update it every couple of weeks, right?

    DB: Yes, that’s correct.

    BL: Do you have a sense of what the update schedule is going to be, or is the CDC not sure yet?

    DB: I’m not sure. I know it was a big haul for them to just get this out, I’m not sure what the next update will be…

    BL: Yeah, well, I’m sure we [Documenting COVID-19] will keep an eye on it. And we’ll tell everybody when it updates. (Editor’s note: As of December 12, it has already been updated! Data now go through November 20, 2021.) So, what are some of the things that you’ve seen in the data from the preliminary analysis that you’ve done so far?

    DB: One of the specific things that I’ve seen, that’s been really important for the work that I’m doing right now, is increases of different types of deaths at home. When people die, they don’t always die in a hospital—they could die in an outpatient clinic, or in an ER, or they could come to the hospital dead on arrival, they could die in hospice, or a nursing home, or at home.

    And one of the awesome things about the CDC data is that you can see, actually, where people have died, and what specific causes of death that those people had when they died. Or, to be precise, you can’t see specific people—but you can see, say, 50 people died of heart attacks in a specific county at home. You would be able to see [in the data] that those people not only died of a heart attack, but they died at home. 

    The takeaway for me has been that respiratory and cardiovascular deaths have increased at home in specific states and counties. Louisiana is one example: it looks like Louisiana has the highest increase of deaths at home from [the CDC designation] “other forms of heart disease,” of any state, at like a 60% increase from previous years. So then we have to ask ourselves, what could lead to that increase? Are people really dying more of heart disease at home, by that much higher of a rate? Or is something else going on here?

    BL: If you were talking to local reporters about this, what would they recommend that they do with the data?

    DB: I would recommend that they take a look at the most recent data, the data from 2020 and 2021, for their area. And also pull some previous years, probably five years [of data], and start looking at causes of death, ages of the people who died, racial and demographic makeup, and place of death. I think different combinations of those data will start to provide some interesting avenues that can lead you to do actual human reporting—asking, what was happening? And why was that happening at this scale?

    The new WONDER data, you can kind-of stretch it and bend it in so many different ways, it can be a little bit intimidating at first. So maybe, it would also be useful to start with a more specific question. If you’re wondering about, let’s say, certain types of deaths in a very specific county. Say you’re wondering if that’s from unintentional drug overdoses, or deaths from respiratory diseases in your county. Then you can start looking at the more granular level of details within those types of deaths—whether it’s racial and demographic makeup, or whether or not the body was autopsied. You can even see the day of the week [that people died]. There’s a lot of different places you can zoom in.

    My overall advice would be: Start with a general question and then explore, then reform that question and explore, then reform that question. The data is both so extensive and so granular that you can get lost in it very quickly.

    BL: You mentioned that it’s very intimidating, which I would second. The first time I looked at the WONDER data, I was like, “What is going on here?” So, what would be your recommendations for working with that data tool? Or any major caveats that you think people should know before they dive into this?

    DB: That’s a great question, because with WONDER, you have to use their querying tool through their website. You can’t really easily and quickly export things or work with an API, though you can export data once you do a query.

    My first caveat would be, keep in mind the suppression of any values under 10. So, that means you can zoom in on certain things, but then you may also have to zoom out. For example, if you wanted to know the leading causes of death for someone, when a body is dead on arrival—if you do that search at a state level, you’ll probably be able to see the first five or so causes before you reach causes that have only happened between one and 10 times, and then that value is oppressed and you can’t see the information. But if you were to do the same search on a national level, you would have a lot more causes for those types of deaths.

    So, I would keep in mind the suppression, when zooming in and out. And also keep in mind, if, say, you’re looking at “dead on arrival” deaths for every county in a specific state, so many causes of death for those [county-level searches] will be suppressed, that your totals from the counties would not match the actual totals [at the state level]. Because you may not be aware that the CDC is not showing you the values that were suppressed if you didn’t click a specific button—or if you’re quickly adding things.

    BL: Another thing that [our team ran into] is occurrence versus residence—that’s something people need to know about. “Residence” means sorting by where people lived, “occurrence” means sorting by where they died. Those don’t always match up.

    DB: Yes, I would say residence versus occurrence is very important to keep in mind, especially because, when you’re redoing a search and scrolling very fast, you can accidentally fill out a state for occurrence instead of residence. Which actually did happen to me, and then I was confused by my own numbers. Then I noticed that there were a bunch of states coming up that I hadn’t meant to search for, because I, like, filtered by residence and then searched by occurrence.

    So yeah, keeping in mind the difference between residence and occurrence is definitely important. Though if you go back in the historical data [before 2018], it’s just residence—just a single state for each death.

    Also, just clear some extra time to get used to working with the WONDER interface. Because, unlike the CDC data updates that are just on the data.cdc.gov website, that you can just quickly download and open up in your technical took of choice—for WONDER, you do have to use the WONDER query site, and it can be difficult to get used to searching and importing. 

    BL: I will say one more thing, while we’re on this topic, that I’ve been doing and that might be helpful for other people: make sure that, if you export data from WONDER, that you always save that notes section it gives you at the bottom [of the exported file]. Because that will tell you exactly what you searched for. So, if you want to replicate something later, you can just go back and look at the notes. I feel like my instinct, often, when I’m looking at a dataset, is to delete all the notes and anything I don’t need—so I have to remind myself, like, “No, you should keep this.”

    DB: That’s actually a really good tip, because I do that… I import the data [to my computer] and then I delete all the notes. That’s a great point.

    BL: Also, what recommendations do you have if people are looking for, like, experts to interview about these data? Say a local reporter wants to search for experts in their area, what should they do?

    DB: I can speak about that, because that’s been really useful for me in my reporting. Once you have this data, or once you’ve researched excess deaths in your area, you should talk with an epidemiologist or a social epidemiologist—someone who would know your state, or maybe even your more local area—about the broader mortality trends in your community. That will really give you a deep understanding of, what were the reasons that people were dying before the pandemic? And what has this expert thought about during the pandemic? And what have they heard, or read, or researched about why deaths are increasing? For example, I talked to two epidemiologists in Mississippi while working on our investigation, and they really helped me understand what I was looking at and looking for.

    BL: Awesome. And then, my last, kind-of big picture question is, why does this matter for people who aren’t epidemiologists or COVID reporters?

    DB: That is also a good question. I think the thing that I have been thinking about over and over again—and it’s something that an epidemiologist told me—which is that, if we understand how people die, then we might know what’s making them sick. And if we know what’s making them sick, then we have a shot at stopping that from happening.

    This data is a very important step in that process, which is learning, in real-time, why people are dying. If we know that, we know what’s making them sick, whether it’s unintentional drug overdoses, or an increase of deaths because of lung cancer or heart disease. Any of those things are important to know, especially in a public health crisis like the one we’re in right now.

    BL: I know we’ve talked before about this sort-of cycle of, what happens when COVID deaths are maybe undercounted in a certain community, and then that contributes to people maybe being less aware of COVID in their community. And then [that lack of awareness] contributes back to the same process.

    DB: Yeah, exactly. I think that’s an important thing as well. Throughout this process—reporting on this topic, and working with this data, and thinking more about death certificates and the information on them—I’ve been increasingly… Not evangelized, exactly, but I’ve seen the light on the importance of that final piece of information of people’s lives. And what it means not only to their families and to the local area and communities, but also what it means when we start pulling that data up to larger and larger groups, and trying to understand: what does this person’s death mean at the level of the county, or the state, or in their racial demographic, or in their age demographic, or by gender?

    All of this is critically important. And it sounds kind-of corny, but in a way, [the death certificate] is like, one really last piece of information that you leave behind for humans after you.

    More national data

  • Public health data in the US is “incredibly fragmented”: Zoe McLaren on booster shots and more

    Public health data in the US is “incredibly fragmented”: Zoe McLaren on booster shots and more

    This week, I had a new story published at the data journalism site FiveThirtyEight. The story explores the U.S.’s failure to comprehensively track breakthrough cases, and how that failure has led officials to look towards data from other countries with better tracking systems (eg. Israel and the U.K.) as they make decisions about booster shots.

    In the piece, I argue that a lack of data on which Americans are most at risk of breakthrough cases—and therefore most in need of booster shots—has contributed to the confusion surrounding these additional doses. Frequent COVID-19 Data Dispatch readers might recognize that argument from this CDD post, published at the end of September.

    Of course, an article for FiveThirtyEight is able to go further than a blog post. For this article, I expanded upon my own understanding of the U.S.’s public health data disadvantages by talking to experts from different parts of the COVID-19 data ecosystem.

    At the CDD today, I’d like to share one of those interviews. I spoke to Zoe McLaren, a health economist at the University of Maryland Baltimore County, about how the U.S. public health data system compares to other countries, as well as how data (or the lack of data) contribute to health policies. If you have been confused about your booster shot eligibility, I highly recommend giving the whole interview a read. The interview has been lightly edited and condensed for clarity.

    Betsy Ladyzhets: I’m writing about this question of vaccine effectiveness data and breakthrough case data in the U.S., and how our data systems and sort-of by extension public health systems compare to other countries. So, I wanted to start by asking you, what is your view of the state of this data topic in the U.S.? Do you think we can answer key questions? Or what information might we be missing?

    Zoe McLaren: It’s the age-old problem of data sources. A lot of cases are not going to be reported at all. And then even the ones that are reported may not be connected to demographic data, for example, or even whether the people are vaccinated or not. Whereas other countries like Israel, and the U.K., your positive COVID test goes into your electronic health record that also has all the other information. 

    And Medicare patients, they have that whole [records] system. There will be information [in the system] about whether they got vaccinated, as well as whether they have a positive test. So that data will be in there. But for other people, it may or may not be in an electronic health record. And then of course, there’s multiple different electronic health record systems that can’t be integrated easily. So you don’t get the full picture.

    But it’s all about sample selection. Not everyone [who actually has COVID] is ending up in the data, which messes up both your numerator and denominator when you’re looking at rates.

    BL: Could you say more about how our system in the U.S. is different from places like Israel and the U.K., where they have that kind of national health record system?

    ZM: When the government is providing health insurance, then all of your records and the [medical] payments that happen, there’s a record of them… And then, because it’s a national system, it’s already harmonized, and everyone’s in the same system. So it’s really easy to pull a dataset out of that and analyze it.

    Whereas in the US, everything is incredibly fragmented. The data, and the systems and everything is very fragmented. The electronic health systems don’t merge together easily at all. And so you get a very fragmented view of what’s going on in the country.

    BL: Right, that makes sense. Yesterday, I was talking to a researcher at the New York State Health Department who did a study where they matched up the New York State vaccination records with testing records and hospitalization records, and were able to do an analysis of vaccine effectiveness. And he said, basically, the more specific, you tried to go with an analysis, the harder it is to match up the records correctly, and that kind of thing.

    ZM: Exactly. It’s easy to match on things like age, sex, race, since everybody has them. But then, the different data fields are gonna have different formats and be much harder to merge together.

    BL: So what can we do to improve this? I know Medicare for All is one option— 

    ZM: Medicare for All, end of story, end of article. It would solve so many problems.

    It’s tricky, though, because there isn’t a simple fix. All of these health systems have their own electronic health records, and integrating them is really costly and hard to do, and who is going to pay for that? There’s also additional privacy concerns about integrating things, in terms of protecting privacy and confidentiality. So, that’s really tricky.

    The way that we get around that, in general, is to have reporting requirements. Like with COVID tests, [providers are] required to report to the CDC or the HHS… Still, that’s also costly and time consuming. But that is kind-of the best thing that we can do right now, is have the different [public health] entities produce reports on a regular basis and send that to a centralized location. And the reports are supposed to be produced in a way that they are harmonized, they’re easy to put together from all the different systems.

    The problem with the different systems not integrating is, it requires everyone to basically fill out the equivalent of a form and send it in—listing individual patient information, or at the state level, individual county information. An example of that is the COVID data. All of the COVID data gets reported up to the national level [by state and county health departments]… 

    But the reporting often gives you the numerators, when you need to figure out the denominators. Because you would want to know, for example, we want to know what proportion of breakthrough cases end up hospitalized. But if only the hospitalized people end up in the data, and a lot of breakthrough cases go either undetected or never tested, or they do an at-home test and there’s no record of that positive case in the system, then your denominator is—there’s a problem with your denominator. That’s a problem with sample selection, you get people that are self-selecting into the numerator [by testing positive], but also self-selecting into the denominator [by getting a test to begin with].

    BL: Yeah, that makes sense. I know you said it would be pretty complicated to basically force different public health departments—to standardize them so that they’re all reporting in the same way. Is there more that researchers in the US could be doing in the short-term to either improve data collection or use what we have to answer questions like, what occupations might confer higher risk of a breakthrough case? 

    ZM: This is a coordination problem. Because in general, we all have an incentive to contribute to having a better understanding of breakthrough cases. But the trick is that, unless the national government or the CDC takes the role of saying what the [data] format’s gonna look like…

    Part of the problem is that there’s an effort involved [in collecting these data] and people don’t want to put in the effort. But if they do want to put in the effort, then you still have a coordination problem, because who gonna to be deciding what format we’re using?

    BL: Or like, what the data definitions are.

    ZM: Exactly. Like, do you report the month and the day of the vaccination dose, or just the month of the dose? Things like that where it doesn’t seem like a big deal, but it does matter for research purposes. If you look, for example, at the Census, or any of the national surveys, like the Current Population Survey or the National Labor Force Survey where we get unemployment numbers, there are big committees that figure out which questions we’re asking and how we ask them. So, if the CDC just says, like, “This is the dataset we’re building,” then everyone [local agencies] will be like, “Okay, we’re gonna send our reports in that way.” 

    Part of [the challenge] is that it takes effort to produce the data, and part of it is somebody needs to coordinate. And usually that would be something the CDC would do, saying, “This is the data that needs to be reported to us,” and everybody reports to them. But they could be doing more, they could be asking for more detailed information—for example, data based on vaccination status, because that information will be important for understanding the progression of the pandemic.

    BL: Yeah. I volunteered for the COVID Tracking Project for a while, and one of the most tedious things that we had to do there was figuring out different definitions for like, what states were considering a case or a test, or whatever else. So that definitely makes sense to me.

    ZM: Exactly. And the COVID Tracking Project filled a gap. Nobody was doing that [collecting data from the states], so the COVID Tracking Project did that… But it’s tricky, because a lot of the stuff that seems like splitting hairs [on definitions] really does make a difference when you’re doing your analysis.

    BL: I also wanted to ask you about what the implications are of this lack of standardized data in the U.S., and the lack of information that we have—largely around vaccinations, but I think there are other areas as well where we’re missing information. So I’m trying to figure out, for this story, how data gaps might contribute to the confusion that people feel when they watch health agencies make decisions. Like watching all the back and forth on booster shots, or thinking about Long COVID, other things like that.

    ZM: Well, we talk about evidence-based medicine, and we also care about evidence-based policy. And so it means that when the quality of data is poor, the quality of our policy is going to be worse. So it really is in everybody’s best interest to have high-quality data, because that is the bedrock of producing high quality policy.

    BL: Right. So if we don’t know, for example, if people who live and work in certain situations are more likely to have a breakthrough case, then we can’t necessarily tell them—we can’t necessarily say, “These specific occupations should go get booster shots.” And then we just say, “Everyone can go get a booster shot.”

    ZM: It means that we’re flying blind. And the problem of flying blind is twofold. One is that you can end up making poor decisions, the wrong decisions, because you don’t have the data. And then the other problem is that you end up making decisions that, in economics, we call it “inefficient.” I think about [these decisions] as, you end up with “one size fits all.” 

    If we have really high quality data, then we’re able to create different policies for different types of people, and that helps minimize any of the downsides. But the less data we have, the more we have to rely on “one size fits all.” And of course, if “one size fits all,” it’s going to be too much for some people and too little for others. Data would help improve that.

    BL: How do you think that this kind of “one size fits all” contributes to how individual people might be confused or might not be sure how to kind of interpret the policies for their own situations?

    ZM: I think in a “one size fits all,” people get very frustrated because they see in their own lives, both the uncertainty and how that can be stressful—and also the waste. The situations where they fall under one policy, but they have enough information to know that that policy doesn’t necessarily apply to them. It does undermine confidence in policymaking. People get frustrated with “one size fits all,” because it seems wasteful.

    Though sometimes the “one size fits all” is still optimal, it’s better than the alternative. For example, the recommendation of “one size fits all” wearing masks tends to trump the “one size fits all” of not wearing masks. But there’s waste. There are situations where we end up wearing masks where they wouldn’t necessarily be needed. And vice versa.

    BL: Yeah. That makes me think of friends I have who are eligible to get booster shots because of medical conditions, but they’re sort-of thinking, “I wish the shots could go to another country where they need vaccinations more.” And that’s not something individuals have any control over, but it’s frustrating.

    ZM: Part of it is, with the booster shots, is the guidelines that say people who have higher occupational exposure to risk [are eligible] without specifying exactly who that is. That is one way that we allow some leeway. So it’s not a “one size fits all” where nobody gets it, because there’s actually people who qualify under higher occupational exposure. But we also don’t want to have a “one size fits all” where we tell everyone they need it, because we do want to be sending doses abroad as well.

    So that’s a situation where we know that a “one size fits all” is not perfect. And so we create a, like, “use your judgement, talk to your doctor” kind-of thing that tries to help people self-select into the right groups… There are likely a lot of people who do have higher exposure and should be getting it, but don’t think the benefit applies to them.

    Editor’s note: According to one analysis, about 89% of U.S. adults will qualify for a booster shot after enough time has passed from their primary vaccine series. And, according to the October COVID-19 Vaccine Monitor report, four in ten vaccinated adults were unsure whether they qualified.

    BL: I also wanted to ask, you mentioned rapid tests—those don’t necessarily get reported. Are there other other things that you think pose data gaps in the U.S. public health system?

    ZM: With rapid tests, the actual tests are not getting reported. But the important thing is, people are getting tested. I mean, the reason we want good data quality is to reduce cases, and we wouldn’t want to limit access to rapid tests in order to collect data, because it’s much easier to prevent the cases by allowing people to get tested in their homes.

    But yeah, just the fact that there’s no centralized database for analysis [is a gap]. I mean, if you look at the U.K., and Israel, they have these great studies, because they’re able to just download, like, the entire population into a dataset. And it has all the information they need, like demographic factors. The fact that the U.S. has made so much of its national policy based on Israeli data, this shows how far behind we are with having our own data to answer these questions.

    BL: Yeah. I know, it’s something like half or a third of cases in the U.S., the CDC doesn’t have race and ethnicity information for [editor’s note: it’s 35%], and other stuff like that. It’s wild.

    ZM: Yeah… And one of the things about reporting is that every additional piece of data you want is very costly. And so you have to be very judicious about [collecting new values].

    BL: Well, those were all my questions. Is there anything I didn’t ask you that you think would be important for me to know for this story?

    ZM: Just that data is helpful for planning now, and helpful for the future. If we can improve our data systems now—it’s part of being prepared for the next pandemic.

    More vaccine reporting

  • Yes, we still need better data on COVID-19 and race: An interview with Dr. Debra Furr-Holden

    Yes, we still need better data on COVID-19 and race: An interview with Dr. Debra Furr-Holden

    Dr. Debra Furr-Holden, photo via Michigan State University

    I recently had the opportunity to discuss data equity with Dr. Debra Furr-Holden, a public health expert at Michigan State University. Dr. Furr-Holden is the university’s Associate Dean for Public Health Integration and Director of the Flint Center for Health Equity Solutions, a health research center focused on Flint, Michigan, where she is based.

    At one of my National Science-Health-Environment Reporting Fellowship training sessions, Dr. Furr-Holden spoke about the Flint water crisis and other health equity issues. Her comments made me think about continued issues in COVID-19 data collection and reporting, so I asked her to discuss COVID-19 data further in an interview for the CDD.

    We talked about the ongoing challenges of collecting and reporting COVID-19 race data, how data gaps fuel vaccine hesitancy, the equity challenges inherent in vaccine mandates, and more.

    The interview below has been lightly edited and condensed for clarity.

    Betsy Ladyzhets: First, I’m curious about your backstory, how you got involved in doing this kind of [health equity] work.

    Dr. Debra Furr-Holden: I think it probably was born out of my own lived experience. My dad died at 37, of a complication from hypertension. My mom died at 56 of an asthma attack.

    It wasn’t until I went to college that I realized that my peers had very different experiences. I went to college with no living grandparents and one living parent, and I just assumed everybody had relatives with, you know, amputated limbs and with diabetes and heart disease. And I realized that’s not the case.

    As I networked with the very small cohort of African-American students in my class, I noticed despite our socioeconomic backgrounds—because I came from sort of more humble beginnings than some of my Black and brown peers—I was like, Oh, [these health conditions are] over-represented in black and brown people.

    BL: How has that informed the work that you’ve been doing with COVID? I saw that you’ve been advocating for better vaccine access and stuff like that?.

    DFH: What I’ve realized is, a lot of what we do around disparities, we do to people, and for people, and on their behalf. But the populations most affected very rarely have a voice, and the solutions that get created and implemented and employed—and we saw it with COVID, we’re seeing it now.

    The President has made a national declaration, give everybody $100 for the newly vaccinated. And that doesn’t make sense to a lot of people. People who are having trouble paying for their hypertension medication or their other things are now being told, we’ll give you $100 to get this COVID vaccine. When earlier in the pandemic, those same people couldn’t get access to a COVID test.

    BL: And in some cases, probably still can’t get access to a COVID test.

    DFH: Yeah. And I’ve just realized, like, my own lived experience that is ongoing still informs my work, but it elevated my authentic and deep appreciation for how important the voice of community and affected populations is in the work. It’s not just about the data. It’s not just about the science… You can only glean but so much from a data table. You need more wind underneath that. And that wind is the voice of community, and the voice of the people that you’re trying to impact and serve.

    So, the big gap to me in our work around how to bridge this gap among the unvaccinated is: we are quantifying who is unvaccinated, but we’re not asking the question of, what is needed to bridge that gap for you to get the vaccine? Instead, I think we’ve got a lot of well-meaning people who are coming up with solutions, but those solutions are not mapping onto people’s concerns. And it’s not moving the needle.

    In Ohio, they offered this big lottery, it did not cause a big boom in vaccination. Same thing is happening in Michigan right now. It did not rapidly accelerate the pace of newly vaccinated people. And because my work is so community engaged, when I talk to people and they tell me the reasons underneath [their vaccination choice], it’s not about the money.

    I call the money the carrot. We’ve tried to dangle the carrot in front of people. That didn’t produce much. Now we’re using the stick.

    BL: The mandates.

    DFH: The mandates, yeah. That will likely produce more [vaccination] than the carrot did, because people will have their hands forced. But that will likely elevate resentment and give way to—any negative consequences or outcomes that come from people being forced into vaccination will likely only further fuel their mistrust of the healthcare system, and our government overall. I just feel like the solutions are not being informed by the people that we’re trying to get on board.

    BL: Yeah. What kind of information do we need to actually inform better solutions, do you think? 

    DFH: We need to hear from the very large and diverse pool of unvaccinated people. Because there’s no one solution here.

    Now, I do believe fundamentally, as a public health professional, I think of public health big population-level interventions that make health choices easy. So things like fluoride in drinking water. We don’t [remember] the time when the cavities and dental cavities were contributing to all of this excess death and morbidity. Why, because we got fluoride in drinking water. So it’s just a non-starter for us now. Same thing for standardized childhood immunizations, which were transformative for eradicating diseases that took millions of lives before we not only developed those vaccines, but made them a part of the standard immunization protocol for children.

    We’ve now got to do the work to figure out how to implement and integrate these COVID protections into our system of care, and have them be more normative. I think all of the mistakes around how the whole pandemic has been handled in the US—how the resources, not just the vaccine, but other resources, like payroll protection, enhanced unemployment, support for essential workers.

    You know, we weren’t providing PPE to essential workers in the beginning. We had national leaders saying you don’t have to wear a mask. All of these things now conflict with, “Oh, we care so much, and everybody has to get vaccinated. Everybody needs to take one for the team.” People just aren’t buying into that.

    BL: They think there’s something else going on, I guess. So, I know, when we were closer to the start of the vaccine rollout, like earlier in 2021, I saw a lot of press attention on the lack of demographic data on vaccinations. A couple of my colleagues at the COVID Tracking Project wrote an article in The Atlantic and there was other kind-of big name publication stuff. But now we still don’t have good data. And it seems like no one is really drawing attention to that. I’m wondering if you have any thoughts on this, and if there’s anything we can do to continue that pressure, because we still do need this information.

    DFH: Yeah, it’s unfortunate, because I always say a lack of data continues to fuel the debate. And the lack of quality data around COVID resources is only fueling the problem. It is an unnecessary and unacceptable omission for providers to administer COVID tests and not collect basic demographic data on the people that they’re testing. It dampens our ability to quantify who is most impacted and what should be the targets of our outreach, engagement, and intervention efforts. And it’s unnecessary and unacceptable.

    In Michigan, the system that we use is called MICR… It would take a programmer about eight seconds to make race, and ZIP code, and gender, and age category a required field to be entered. And we just simply haven’t done it. And so as a result, it’s hard for us to quantify the extent of a problem.

    Because, remember, COVID cases are only a function of COVID testing. You can only get identified as a COVID case as a function of having a COVID test. If you’re in a household, and there’s a known case in the household, and all of the other [household] members display classic COVID symptoms, if they don’t get a test, they don’t get counted anywhere. So we know that we’ve greatly underestimated the extent of the problem.

    BL: When I asked you about this at the SHERF session, you mentioned that there’s a provision in the CARES Act that requires providers to do this [data collection]. Can you talk more about that? And what we can do to actually have some accountability there?

    DFH: Yes. There is a provision in the CARES Act that all COVID testing providers have to collect these core demographic variables. And then there was follow up guidance that was issued. And when the new administration took office, they haven’t enforced that [guidance].

    So COVID testing providers continue to receive these resources to provide COVID testing, with no quality assurance or quality control, to ensure that they’re actually collecting and entering that demographic data. It then shifts the responsibility to backfill that information to local health departments and other providers, to try to link insurance records or electronic health records. Or even worse to do outreach and contact tracing and actually contact cases, by phone or by email to try to backfill that information. When there are so many other competing demands, it’s an unfair and undue burden to place on an already overstressed segment of our healthcare system. 

    What it’s akin to is gums without teeth. We have the law, but there’s no enforcement or compliance checks to ensure that that law is being honored. And I think a simple solution is compliance checks. We need compliance checks, and we need enforcement.

    BL: Do you have any thoughts on other stories that we should be telling? Like, what should I tell my journalist friends to cover around COVID and health equity?

    DFH: One thing is probably already on your radar, which is the fact that we’re not doing systematic genetic sequencing on current strains of COVID. So it’s hard to estimate, you know—people keep talking about the Delta variant, but we have thousands of variants of SARS-CoV-2 now. And we just don’t have a good system for genomic surveillance to understand them.

    And the CDC a few weeks ago said, we’re just going to stop doing the genomic sequencing on any kind of systematic level and reporting. It’s a problem, because with breakthrough cases, and

    the vaccinated now showing up in hospitals and emerging data saying that even if you’re vaccinated, you can still spread and transmit… I just had a conversation with somebody who works in our building who said, I don’t want to get vaccinated, because if I get COVID, I want to have symptoms, so I’ll know, so I can protect my nine-year-old who’s got asthma. Like, I want to know. A lot of people now feel like the vaccine increases the chances of them being an asymptomatic carrier.

    We just really have to collect data. Instead of mandating shots in arms, we should be mandating the data so that we have better information and can do more credible and transparent information dissemination to communities.

    BL: Yeah, so that we can actually answer people’s questions on these things.

    DFH: Yeah.

    BL: I was also wondering if you had any recommendations, either of good stories that do a good job of covering these issues we’ve been talking about, or data sources or resources that myself and other journalists in this space should be paying attention to.

    DFH: We should be putting the press on the CDC to collect and compile the data. Like, the data on cases, all of that data should be disaggregated by race. And the percentage of cases with unknown race or unknown gender or unknown geography should also be reported. Because I don’t know if people notice this, but a lot of times [the CDC is] presenting data only on cases with complete information. But the missing information points to something important as well.

    BL: I think it’s something like they have maybe 50% or 60% of cases with known race. But where’s that other share of cases? [Editor’s note: It’s 63%, as of August 14.]

    DFH: The assumption is that the distribution of these variables in the unknown is similar to that of the known. But it is a major assumption. And it’s not an assumption that we should be making.

    BL: I see. Yeah. Anything else [you’d recommend as a story idea]?

    DFH: I do like this carrot stick analogy. The carrot is not working, the dangling the big incentive is not working. The stick will likely work. If you tell people, “You can’t get on a plane, if you’re unvaccinated,” there will be a lot of people who are unvaccinated right now who will get vaccinated because they’ll not want to lose the opportunity to travel.

    Think about the media. If you are chasing a story, or if you’ve got to be on site for something… If you’re in New York and you’ve got a story in California, you’re not going to drive to California, you will likely get off the fence and get vaccinated.

    I feel like a larger problem is, we have to engage experts in the work to make sure that we’re not furthering inequity [with mandates]. Because if we use, now, the stick, and start to mandate it…. [Michigan State University] has now mandated vaccination for all faculty, staff and students who want to return to campus by September 7. I know that that will likely produce greater increases in vaccination than did the incentives of cash payments, or lotteries or other things.

    But we have to keep an eye toward equity, and make sure—what if there’s disproportionality and then who does that impact? Are we going to see an increase in Black and brown people, or people with disabilities, or people with chronic health conditions, losing their jobs, or dropping out of school, or some of these other things? There just needs to be more thoughtfulness to how we apply these policy interventions to make sure that it’s not furthering inequity.

    BL: Have you seen any examples of where that’s been done successfully?

    DFH: No, because it’s all just coming out now.

    BL: I know there are some places, like in New York, they’re giving you an option, saying, “You can get vaccinated or you have to be tested once a week.” Is that effective? Or does that still fit into what you’re talking about?

    DFH: I think we’re gonna figure that out. And if that’s the case, then again, we gotta deal with the access issue, and people need to have fair and equitable access—and affordable access—to COVID testing.

    BL: Yeah, totally. And the last kind of big question I had for you: one thing I think a lot about as a journalist who is still rather early-career and has been covering COVID very intensely is that this is probably just the beginning of us dealing with major public health crises. You know, continued climate disasters and all that stuff.

    And I’m wondering how you think about preparing for the next COVID, or the next whatever it’s going to be. What lessons do we take from these past couple of years?

    DFH: Well, I think we’ve learned there is a business case for preparedness, and a business case for equity. Our lack of preparation for this pandemic will have cost our country tremendously. There’s going to be tremendous financial toll. So, there’s a business case to be made for preparedness.

    We learned that with the Flint water crisis. Not having the million-dollar investment in the water treatment system, not spending the 150 bucks a day on anticorrosives, those things will have cost us hundreds of millions of dollars to now replace and repair the whole water infrastructure system and pay settlements from the Flint water crisis.

    And then there’s also a business case for equity. Not doing a better job of equitably rolling out the vaccine early on caused a lot of people who were a “yes” to sort of say, “why bother?” And now many of them are a “no.” These are people who earlier on [were amenable], but then all these reports come out and get sensationalized by the media of side effects and blood clots and heart inflammation. And so a lot of people who were in line, trying to move through the line to get vaccinated are now an absolute “no.”

    That’s going to cost us as well, because we have fallen well short of that 70% goal. And new vaccinations are moving at a snail’s pace. So I think what we’ve learned—and we’ll really know, the impact of it in the next few years—is not being prepared and not practicing equity will have a tremendous financial toll on the country.

  • A dispatch from Provincetown, Mass.

    A dispatch from Provincetown, Mass.

    Provincetown in June 2006. Source: ingawh via Wikimedia Commons

    Last week, a COVID-19 outbreak in Cape Code, Massachusetts was revealed to be the subject of a major CDC study providing evidence of the Delta variant’s ability to spread through vaccinated individuals. The outbreak quickly became the subject of national headlines, many of them sensationalizing Delta’s breakthrough potential—while failing to provide much context on the people who actually got sick.

    Here’s one big piece of context. Provincetown, the center of this outbreak, is one of America’s best-known gay communities, and the outbreak took place during Bear Week. Bear Week, for the uninitiated, is a week of parties for gay, bisexual, and otherwise men-loving men who identify as bears—a slang term implying a more masculine appearance, often facial and body hair.

    This week, I had the opportunity to talk to Mike, a Bear Week attendee from Pittsburgh who caught COVID-19 in Provincetown. (Mike asked me to use only his first name to protect his privacy.) He told me about his experience attending parties, getting sick, and learning about the scale of the outbreak.

    We also discussed how Provincetown and the Bear Week community were uniquely poised to identify this outbreak, thanks to a better-than-average local public health department and a group of men who were willing to share their health information with officials.

    The interview below has been lightly edited and condensed for clarity.

    Betsy Ladyzhets: My first question is just like, how are you doing? How have you been after being involved in this outbreak?

    Mike: I’m good… I live in Pittsburgh, I drove back on that Saturday [after the week of Provincetown events] and on Sunday, I started coughing really bad as I was driving home. This just came out of nowhere. I had to pull over, I’m like, yeah, I’m not good. This cough was a lot worse than I had anticipated. So, that was my first symptom. I went into the office Monday after getting home…  My first test was negative, on like Monday or Tuesday. But like, I’m still coughing. I didn’t fully trust it. So I got another one Friday, a PCR test.

    BL: So, you got tested twice? Did you experience contact tracing, or how did you get identified as part of the outbreak?

    M: I mean, I just knew I’d been there. Um, no one reached out but… There was a Facebook group, probably ten or fifteen thousand people in it. Lots of people posted about their test results. Like, people after they were leaving [Provincetown], started quarantining.

    The thing about Provincetown is, there were events that happened in the first week [of July, for July 4] that no one really had time to process… Then Bear Week, the week I went, I went at the busiest week of the year for the town. And it had to be, from a planning perspective, I don’t know that was necessarily the best time to have two huge events back to back.

    All the official events for the week that I went were canceled, though there were some of the regular bars and stuff doing events. There was, at the time, I think one venue that has a mostly outdoor party every day from like three to seven, that was very heavily attended with one or two thousand people every day, mostly outside and it’s possible to distance at. I only ended up going once or twice just because it wasn’t really where I wanted to be regardless of COVID risks, it wasn’t particularly a scene that I was craving at the time.

    I only went to, maybe, three or four indoor things the whole time, and it was without a mask for two or three of them. There’s a bunch of nightclubs in Provincetown that were still having events. And I don’t think that any of the bars themselves that were having events were requiring vaccination cards or anything. One venue that I saw a show at, they announced the next day that they were making either masks or proof of vaccination required. One of the venues that has outdoor events, they just moved all their shows outside instead of inside.

    BL: I see. And you mentioned the Facebook group, was that how you found out that a lot of people were getting tested and things like that?

    M: Yeah, there were somewhere between ten and fifteen thousand people in the group, planning this whole week. People usually come to Provincetown from all over, sometimes from abroad, though I don’t think there were many people coming from abroad this year because of the restrictions.

    BL: How did you learn about the big CDC study getting written about this?

    ML: I didn’t really have any idea until afterwards. There were lots of people in the group saying that Barnstable County, or the Massachusetts Department of Health, wanted to know—they wanted people to call if they’d gotten a positive test so they could keep better track of it. I mean, I think part of why the report was able to happen was that it was in a place with better respect for public health than, like, the state of Florida would have, if this kind of outbreak would’ve happened there.

    B: Yeah, I mean, it definitely seems like they responded quickly. Because I know they had, like, a 15% positivity rate one week, and then within a pretty short time it was back down.

    M: The town itself is a mostly gay, retirement-somewhat community. They can spend lots of money on other things [like public health]. They’re not necessarily spending money on schools because of how many people don’t have any kids around that they need to spend money on. And I mean, there are a lot of residents who live there year-round who tend to be older and are at more risk.

    So the week [Bear Week] itself is unique, and then there was a huge community presence about it, everyone wanted to be—for the most part, we’re comfortable about reporting afterwards. I don’t think anyone knew, walking into this, what it would lead to, but… there’s a feeling of community, and that ten thousand-ish Facebook group, I don’t think we otherwise would have necessary talked to each other or told each other about Massachusetts [public health department] asking people to call if they were positive.

    BL: And did you do that? Did you call them?

    M: Um, I personally didn’t, since I didn’t even find out I was positive until a few days later.

    BL: Now, as you know, this outbreak has gotten a lot of national coverage, it’s been kind of sensationalized, with a lot of people focusing on the vaccine breakthrough cases and stuff like that. I know you were not personally one of the people whose test measurements are included there. But what is that experience like of being part of this thing that has gotten so much national attention?

    M: I posted about it on social media and there were lots of people who were surprised or whatnot. I think, at least in my head, I went in with a calculated risk, of like 10, 20, 30, or more in the ten thousand-ish people coming, a lot of them are traveling on planes. I drove, thinking I’ll come into this place and I think I’ll make okay decisions…

    And there were people in this one place for a whole week, that I guess you were able to test from the CDC’s perspective. I don’t think there are many other places that are as remote as Provincetown where people are staying for the entire week, and everyone generally leaves on the same day, and everyone was in conversation with one another, talking about what happened.

    Related links