Investigative journalism – In February, Federal Health Minister Greg Hunt boasted that AstraZeneca’s COVID-19 vaccine offered “100% protection” against death in the primary analysis of phase III trials.
The published study in The Lancet, however, revealed a more nuanced picture.
In the trial of 23,848 subjects across the UK, Brazil, and South Africa, there was one death in the placebo group and no deaths in the vaccinated group.
One less death out of a total of one, indeed, was a relative reduction of 100% but the absolute reduction was 0.01%. (1/11,724 – 0/12,021)
Similarly, in February the CDC director Rochelle Walensky co-authored a publication in JAMA, which stated unequivocally:
“Clinical trials have shown that the vaccines authorized for use in the US are highly effective against COVID-19 infection, severe illness, and death.”
However, there were too few deaths recorded in the controlled trials at the time to arrive at such a conclusion.
The 6 month follow up data from the blinded Pfizer trial found there were 15 deaths in the vaccine group and 14 deaths in the placebo group. (see table S4)
Last week, at a roundtable meeting in the US Capitol, Prof Peter Doshi, associate editor of The BMJ raised concerns about the statements made by the CDC director.
“The trials did not show a reduction in deaths, even for Covid deaths. The evidence was flimsy,” said Prof Doshi.
“Those who claimed the trials showed that the vaccines were highly effective in saving lives were wrong. The trials did not demonstrate this.”
Prof Doshi was not passing judgement on the vaccines. Instead, he was critical of the way authorities had portrayed trial data to the public.
All the public announcements about the vaccines were initiated by the vaccine manufacturers in highly curated press releases, and it significantly shaped the public narrative, setting the stage for high expectations.
For example, Pfizer published a press release claiming the vaccine was “95% effective against COVID-19.” Several weeks later, the actual trial results were published in the New England Journal of Medicine.
- In the vaccine group, 8 out of 18,198 people had COVID-19 symptoms (0.04%)
- In the placebo group, 162 out of 18,325 people had COVID-19 symptoms (0.88%)
The vaccine reduced the baseline risk from 0.88% down to 0.04% after two months. That is, a ‘relative risk reduction (RRR) of 95% but an absolute risk reduction (ARR) of 0.84%.
Hence, if someone’s baseline risk of COVID-19 is very low to begin with (as it is for most people under 50 years), a 100% reduction in risk is trivial.
An editorial in The Lancet compared the RRR of each vaccine to the ARR:
Notably, when quoting the vaccine’s harms, authorities will use the smaller percentage, ARR, presumably to minimise public concern about adverse events.
If authorities are using different metrics to convey the harms and benefits of a medical therapy, it is misleading the public.
Exaggerate the benefits?
It is well established that only quoting RRR without quoting the ARR, can inflate or exaggerate an intervention’s effect size and clinical importance, as well as increase people’s willingness to receive the treatment.
It has been referred to as the first “sin” against transparent communication by Gerd Gigerenzer, director of the Harding Centre for Risk Literacy at the Max Planck Institute. He says it can be used as “a deliberate tactic to manipulate or persuade people.”
“Many physicians, patients, health journalists and politicians do not understand health statistics. This collective statistical illiteracy has resulted in serious consequences for health,” Gigerenzer says.
John Ioannidis, Professor at Stanford University and the most cited physician scientist, agrees.
“In my experience, innumeracy is widely prevalent,” says Prof Ioannidis.
“This is not happening just for vaccines. Over many decades, RRR has been the dominant way of communicating results of clinical trials. Almost always, RRR looks nicer than absolute risk reductions.”
When asked if there was any justification for misleading the public about the vaccine’s benefits to encourage uptake, Prof Ioannidis rejected the notion.
“I don’t see how one can increase uptake by using misleading information. I am all in favour of increasing uptake, but this needs to use complete information, otherwise sooner or later incomplete information will lead to misunderstandings and will backfire,” says Ioannidis.
The way authorities have communicated risk to the public, is likely to have misled and distorted the public’s perception of the vaccine’s benefit and underplayed the harms.
This, in essence, is a violation of the ethical and legal obligations of informed consent.