On April 13th, the CDC & FDA paused the administration of the J&J vaccine for Covid-19 after six women who received the vaccine developed rare blood clots, fatal in one case [StatNews]. The pause was lifted ten days later, albeit with a warning of rare side effects in women under 50 [CDC].
This decision came in for massive criticism. It was (correctly) pointed out that the overall mortality and health risk of Covid-19 far outweighs the risk of the vaccine [WBUR]. Economist/baby-whisperer Professor Emily Oster harshly critiqued the CDC’s decision, suggesting that “we have a hard time parsing what is the risk of anything” [Washington Post]. This chart from Daniel Bier (h/t Marginal Revolution) suggests the decision may have amped up vaccine hesitancy and caused vaccine uptake to plummet:
However, a review of the data that centers on the idea of “base rates” suggests that the decision was actually prudent, when considered narrowly. This is a form of the “extension neglect” cognitive fallacy [Wikipedia], which fails to consider what the appropriate comparisons really are. When making a well-balanced comparison, the relative risks no longer seem so crazy - and the real pressure on cost-benefits moves to the second-order effects like how the decision shapes public opinion about vaccines.
What the Science Said
The key question here is what are the appropriate risks to compare. The articles cited above, in the Washington Post and WBUR, make the case succinctly (from the Post):
…the fatality rate from the coronavirus is extremely low for people in their 20s and 30s, who have about a 0.2 percent chance of dying of it. But even those low odds mean that for every million people in that age group who contract the virus, 2,000 people will die of it. The odds of dying increase somewhat for those in their 40s.
That’s clearly a higher fatality rate than what has been observed so far with the Johnson & Johnson vaccine. Of the six women who developed dangerous blood clots, one has died.
However, this is where we run into trouble. This analysis, while sensible, makes an assumption that is probably unwarranted - that the risk from the vaccine is evenly distributed across the population. This may be the case, in which case the decision is a no-brainer. However, if there are good reasons to believe that the mechanism behind the clotting only applies to a certain subpopulation, e.g., women under 50, then we should only be comparing rates in those groups. By including other groups more vulnerable to Covid, and without negative vaccine side effects, we may end up too lightly evaluating risks vs. rewards.
Kaiser Fung at Big Data, Plainly Spoken [link] walks through the data and concludes that for the specific population of women under 50, the risks of the J&J vaccine and Covid-19 may actually be comparable. The key insight is that key risk factors are both age and gender (Covid-19 is more dangerous for men) and so death from Covid-19 for this population is a rare event. Very rare, as it turns out:
The upshot is this, as Fung writes:
among females aged 18-50…the chance of dying from Covid-19 is about the same as from dying from blood clots+low platelets after taking the J&J vaccine.
With that in mind…the CDC’s risk calculus, and understanding of basic probability, comes off rather differently, doesn’t it? This is a topic I must confess I didn’t initially understand, and in hindsight feel rather embarrassed for jumping to a similar conclusion as those most critical of the decision.
What the Science Can’t Say
The CDC did a terrible job explaining this decision. This is a self-evident claim, because a decision that cannot be understood or parsed by the public indicates by default a failure of public communication. The most persuasive critiques of the decision center on this failure of public communication, and specifically whether it led to an increase in vaccine hesitancy by fueling fears that all vaccines were unsafe. For the record, this fear seems unfounded [Kevin Drum]. Defenders make the opposite claim, which is that being maximally risk-averse the CDC will maximize public confidence in the vaccines.
If the real stakes are a matter of second-order public opinion rather than first-order management of vaccine risk, this seems like the exact type of judgment the CDC is unqualified to make. If the Twitterati are out of their lane in evaluating vaccine efficacy, epidemiologists and virologists are probably out of their lane in evaluating the second-order impacts of decisions on public opinion. I hate to talk my own book here as a (failed) political scientist but if the most important issue is public opinion, the CDC ought to be consulting with political scientists who actually study how the public forms opinions. Taking this concern more rigorously might prevent future fiascos like the CDC’s misguided campaign against masks in 2020 [Wired].
It also shows why “trust the science” isn’t an ultimately satisfying approach to decision-making about pandemic management. The CDC made a reasonable judgment here about reasonable risks of a novel drug, but it just lacks the ability to evaluate the second-order consequences of those decisions and what is ultimately best at a society-wide level. It also lacks any real democratic legitimacy to make those type of society-wide decisions, which ultimately have to lie with elected officials. Politicians saying to “trust the science” are ultimately trying to dodge their own central responsibility.