Subscribe to Second Opinion for a weekly roundup of health and medical science news.
A Canadian study that vastly underestimated the protection COVID-19 vaccines provide against the Omicron variant is being revised — but not before it spread widely on social media by anti-vaxxers, academics and even the creators of the Russian Sputnik V vaccine.
The Ontario preprint study, which has not yet been peer reviewed, suggested that any three doses of mRNA COVID-19 vaccines were just 37 per cent effective against Omicron infection, while two doses actually showed negative protection.
The preprint has been shared on Twitter more than 15,000 times in the two weeks since it’s been published, according to Altmetric, a company that tracks where published research is posted online. That’s in the top five per cent of all research it’s ever tracked.
The group behind Sputnik V shared the results to its one million Twitter followers earlier this month, saying the study showed “negative efficacy” of two mRNA vaccine doses and “quickly waning efficiency” of a booster. The group did not respond to questions from CBC News.
Dr. Vinay Prasad, an associate professor of epidemiology at the University of California-San Francisco, also shared it on Twitter, asking why the U.S Food and Drug Administration (FDA) and Centers for Disease Control (CDC) would advise a booster for Omicron at all.
Study updating findings with totally different results
But the paradoxical findings were later found to have been influenced by behavioural and methodological issues, such as the timing of the observational study, the way in which vaccine passports altered individual risk and changes in access to COVID-19 testing.
The results are currently being updated with additional data that showed completely different results, said Dr. Jeff Kwong, the study’s lead author and an epidemiologist and senior scientist at the Institute for Clinical Evaluative Sciences (ICES) in Toronto.
“We’re in the process of adding two more weeks of data and it looks like there’s no more negative VE (vaccine effectiveness). Our results are now more in line with the data from the U.K. where it’s lower, for sure, compared to Delta, but never getting to negative,” he told CBC News.
“And then higher VE with the boost. So I think that’s good news and we’re just in the process of running those analyses and we hope to have an updated version, a version two, by sometime next week.”
A recent report from the Imperial College London COVID-19 response team found that while Omicron largely evades immunity from prior infection and two doses provided just zero to 20 per cent protection, three doses increased that to between 55 and 80 per cent.
That means the updated preprint could end up showing that protection against Omicron infection is more than twice as high as initially reported. As of Friday, the preprint study remained unchanged on the medRxiv website where it was posted.
CBC cited the study in an analysis story last week, but has since removed reference to it until the data is updated.
The study was also highlighted by the federal government’s COVID-19 Immunity Task Force earlier this week, before the discrepancies in the data were discovered.
“We’ve touched base with Dr. Kwong and indeed he informed us of new data as of Monday night,” a spokesperson said in response to CBC News raising concerns about the study’s accuracy.
“As the data from this week does change things, we’ve pulled the preprint from our magazine that’s being sent out today.”
Dr. Danuta Skowronski, a vaccine effectiveness expert and epidemiology lead at the B.C. Centre for Disease Control, who developed the vaccine study design used in the preprint, posted a commentary urging “extreme caution” with the results last week.
“If you have a negative estimate, you want to start looking at, OK, well, which subgroup is driving that and why?'” she told CBC News.
“Is it the asymptomatic? Is it the symptomatic? Is it people who were screened for work? Is it people who had a rapid antigen test? Which group is it that’s driving that paradoxical finding?”
Skowronski said until those questions have been resolved, “all bets are off” on the interpretation of the results and “the validity of the study has to be questioned.”
“In the real world, we cannot control the behaviour of people, and so these studies are susceptible to lack of comparability between the vaccinated and the unvaccinated,” she said, adding that vaccine passports dramatically changed the risk of exposure in Ontario.
“There are good reasons to believe that the very slim fraction of people who remain unvaccinated — that group are quite different now from vaccinated individuals.”
Study spread like wildfire with anti-vaxxers online
The study highlights the speed in which early study results that have not undergone peer review can spread online in the pandemic and the way in which inaccurate findings can be weaponized to fit an agenda before they can be corrected.
Many who shared the study on Twitter used anti-vaccination rhetoric to allege boosters didn’t work against COVID-19, while others posited the vaccines should not have been approved for emergency use by the FDA in the first place because they did not meet its initial 50 per cent efficacy threshold.
“This will definitely be used by bad actors to consolidate support for their views about the lack of COVID-19 vaccination effectiveness,” said Ahmed Al-Rawi, an assistant professor at Simon Fraser University’s School of Communication who specializes in disinformation.
“I would immediately take it down and make some public statements about the inaccurate findings of the study, because this has been shared widely on social media and it will only confuse people more.”
WATCH | Ontario ICU overwhelmed by mostly unvaccinated patients:
Mostly unvaccinated patients are overwhelming the ICU at a Sarnia, Ont., hospital and some will head home with a new perspective on COVID-19, the vaccine and life. 3:39
The study also notably did not look at the protection vaccines offered from severe COVID-19, which has been shown to be much higher than against Omicron infection alone — something Kwong says he and his colleagues will be adding in a future version.
While COVID-19 vaccines don’t provide total protection from infection, they do work well at preventing serious disease. New data from the Public Health Agency of Canada found Canadians with two doses were 19 times less likely to be hospitalized than those unvaccinated.
“Several studies have shown modest protection from two doses against Omicron infection, but better protection against severe outcomes such as hospitalization,” said Marc Lipsitch, an epidemiologist at the Harvard T.H. Chan School of Public Health in Boston.
“This benefit is over and above any possible benefit of preventing infection or transmission.”
Lipsitch said Skowronski’s criticisms of the study are valid. He has cautioned against comparing positive cases among those with symptoms with those not tested for different reasons, adding he very much agrees this approach can be a source of “substantial bias.”
“When investigators try to share early results in the interests of public health, as these folks did, there’s often a lot of uncertainty in those estimates,” said Dr. David Fisman, an epidemiologist at the University of Toronto’s Dalla Lana School of Public Health.
“But it’s very hard to reel in once people start using early versions of your work in support of misinformation.”
Skowronski said the rapid sharing of COVID-19 vaccine studies on social media has completely changed the research landscape, adding more pressure to get early results right.
“You need to ask yourself, why do we need to post it now? Why can it not wait the one or two weeks? How will this impact public and policy decision-making?” Skowronski said.
“And if you can’t answer that, then we really should be asking ourselves: Why are we rushing to preprint?”
WATCH | Canadians urged by health experts to take first available vaccine:
Health experts across the country are urging Canadians to stop shopping around for their preferred brand and take whichever COVID-19 vaccine is available. 2:13
Skowronski released a study in 2010 showing paradoxical negative vaccine effectiveness during the 2009 H1N1 pandemic that found those who had a flu shot were more likely to get infected with the influenza strain than people who hadn’t, which was later proven right.
But she first assumed the findings were methodologically inaccurate, reached out to outside experts around the world, conducted multiple different studies and worked with an international panel of experts.
“I learned the lesson the hard way back in 2009 in dealing with paradoxical findings and the level of rigour required,” she said. “You don’t approach this in a casual way — it does require lots of thinking, lots of worry — before you can arrive at this.”
Dr. Ivan Oransky, co-founder of Retraction Watch, a website that tracks errors in science journals, said because the study turned out to be “flawed,” the researchers should move fast to update their findings.
“They’re doing the right thing. The question is how quickly will they do it?” he said. “I mean, they’re talking about next week … but that is a bit of an eternity in this day and age.”