Cochrane Collaborative concludes e-cigs as medical interventions help smokers quit (again) while continuing to ignoring stronger more relevant real world evidence that they don’t

The Cochrane Collaborative, an organization that does systematic reviews generally to guide clinical practice, released a “Electronic cigarettes for smoking cessation (Review).” Like earlier Cochrane reviews, this review focused on randomized clinical trials that are designed to study effectiveness of medicines delivered under clinical supervision, not e-cigarettes as consumer products used in the real world without clinical supervision.

Nevertheless, the BMJ news story made it sound like the new 316 page report was a big deal:

Researchers from the University of Oxford examined 78 studies, including 40 randomised controlled trials, which involved 22 052 adults who smoked to assess the effectiveness, tolerability, and safety of e-cigarettes to help quitting. They found that quit rates among adults using nicotine e-cigarettes were higher than those receiving nicotine replacement therapy (risk ratio 1.63, 95% confidence interval 1.30 to 2.04; I2=10%; six studies, 2378 participants). This could translate to an additional 4 quitters per 100 (95% CI 2 to 6), the researchers said. [emphasis added]

But, in reality, there is nothing new here.

While it is true that the Cochrane reviewers considered 78 studies, the conclusions about e-cigs and quitting came down to just six studies, summed up on page 261:

More important, their conclusion that the relative risk for cessation is 1.6 is virtually identical to the 1.6 relative risk we reported two years ago:

(The slight differences in the estimates — 1.62 (95% CI 1.29 , 2.04) in the Cochrane review and 1.56 (95% CI 1.17-2.06) we reported — are well within the “margin of error” (95% CI) of each other.

At this point, there are probably more meta-analyses of these few RCTs of e-cigs as medicines for smoking cessation than there are studies to analyze.

More important, while RCTs are the gold standard for evaluating medicines, but they are not useful when assessing population effects of consumer products.  The population studies show no effects of e-cigs as consumer products on cessation. 

Here is the result of our meta-analysis of e-cigarettes and quitting smoking in the real world, which is based on a lot more information (44 studies):

The odds of quitting are not significantly different from 1.0, indicating no cessation benefit for e-cigs in the real world.

Since we published our meta-analysis, Linnea Hedman and colleagues published an updated meta-analysis of the population studies limited to longitudinal studies that followed the same people forward in time, the strongest kind of epidemiological study. They didn’t find any cessatio benefit, either:

And the more recent population studies that are long-term (3 year) longitudinal studies of the FDA PATH data are even more troubling.

• Olatokunbo Osibogun and colleagues from Florida International University found that that smokers who want to quit are much less likely to actually quit if they use e-cigarettes than if they don’t use e-cigarettes.

• Ruifeng Chen and colleagues at UCSD drew the same conclusion based on a different 3 years of PATH data:

What the relevant evidence consistently shows is that e-cigarettes as consumer products do not help smokers quit cigarettes and, over a longer time, keep them smoking.

It is also telling that, to date, no tobacco company has submitted RCT data (what Cochrane keeps re-analyzing) to drug approval authorities anywhere in the world as part of an application to approve e-cigarettes as clinical cessation medicines.

But, until they do (as we discussed in detail), analyses such as what Cochrane has done, despite its length, don’t really provide any useful information to the FDA Center for Tobacco Products and other regulators around the world who are charged with making decisions about e-cigarettes as consumer products.

The Cochrane authors did identify a single study comparing varenicline with ecigs for cessation (Analysis 2.1, page 268); varenicline tripled the odds of cessation compared to ecigs (OR 0.31; 95% CI 0.11 , 0.82, favoring varenicline). The authors don’t mention this interesting result in the abstract.

They also include many analyses of adverse events and physiological changes associated with e-cigarettes observed in these trials. The problem with all these analyses is that the follow-up time and sample sizes are too small to make meaningful statements about the health effects of e-cigarettes. This is another area in which large population studies are the key to understanding the actual health effects of e-cigarettes.

I’m amazed that Cochrane keeps wasting its time publishing essentially the same meta-analysis of the same small number of small RCTs while ignoring the much larger and more relevant population observational literature on what happens to smokers in the real world who use e-cigarettes.

Physicians for a Smokefree Canada also has an excellent discussion of the Cochrane review, which is available here.

I posted a rapid response making similar points on the BMJ website. As of 26 Nov 2022 there had been no response to these points/