In 1971 Scottish physician Archibald Cochrane published a short book, Effectiveness and Efficiency: Random Reflections on Health Services, that made the point that most medical and surgical therapies in common use had never been shown to work. He argued that for the use of randomized controlled trials (RCT) in which people in the trial were randomly assigned to the therapy being assessed or a control group who received a placebo (an inactive substance that resembled the therapy being evaluated) or conventional therapy was the only reliable way to assess the value of a therapeutic intervention.
Cochrane’s ideas led to widespread acceptance of RCTs as the gold standard for assessing therapies and creation of the Cochrane Collaboration to promote objective assessment of therapies, including publishing systematic reviews of therapies that have become very influential.
It is important to understand this history because the Cochrane collaboration’s rules were developed to assess therapies, which are typically a specific intervention (typically a specific drug or surgical procedure) applied using a specific dose to a specific patient population.
That is the information that needs to be submitted to win approval of therapies from agencies like the US FDA.
The new Cochrane review of e-cigarettes as smoking cessation therapy
On October 14, 2020, the Cochrane Database of Systematic Reviews published Electronic cigarettes for smoking cessation that concluded that e-cigarettes were more effective than nicotine replacement therapy for smoking cessation. (This was an update of an earlier Cochrane review that drew the same conclusion.)
The 193 page report followed Cochrane’s rigorous standards for assessing therapies that are grounded on the view that randomized controlled trials are the best way to assess therapies. As a result, although the review assessed 50 completed studies, representing 12,430 participants, including 26 RCTs, the final analysis that led to the conclusion that e-cigarettes were superior to NRT was only based just three studies involving 1498 participants (Bullen 2013, Hajek 2019, Lee 2018).
In all of these studies free e-cigarettes were provided to participants as part of a clinical smoking cessation intervention. So, the question that the Cochrane review answers is, “When delivered to patients as part of a smoking cessation intervention, are they associated with more quitting than NRT?”
A positive answer to this question would support the use of e-cigarettes as smoking cessation therapies and suggests that an e-cigarette company might be able to win approval for e-cigarettes as a cessation therapy if they applied to FDA with similar RCT data for their specific product used in a specific group of patients in a specific way.
To the best of my knowledge, no e-cigarette company has applied for official approval of e-cigarettes as smoking cessation therapies. Of course, there is also be the issue of what the associated risks are; whether e-cigarettes as cessation aids would have a favorable benefit/risk ratio is a separate question. There is also a high likelihood that if e-cigarettes were approved as smoking cessation therapy, they would only be available by prescription due to their high addiction potential.
The Cochrane review is not relevant for assessment of the effects of e-cigarettes as actually used as consume products in the general population
The fundamental problem with relying on this review to assess the impact of e-cigarettes on smoking cessation is that, for almost all users, e-cigarettes are consumer products not supervised clinical therapies.
Assessing the effects of e-cigarettes on quitting smoking in the real world one needs to look at population-level observational studies. Such studies are (appropriately) considered low quality and are excluded when assessing clinical efficacy of e-cigarettes as cessation therapies under the Cochrane protocol, which is designed to ask a different question (therapeutic efficacy). But they are precisely what one needs to consider when assessing e-cigarettes as consumer products.
As a result, regulatory authorities such as the FDA Center for Tobacco Products, which regulates e-cigarettes as consumer products, and local and state legislators considering e-cigarette or flavored tobacco product sales bans should ignore the Cochrane review (and other similar analyses) in their deliberations and instead rely on population studies which show no cessation benefit.
A couple other points
Simon Chapman published an excellent commentary on the Cochrane report making the point that, even taking it at face value, the absolute cessation benefit for e-cigarettes (and NRT) is small.
Hayden McRobbie and Chris Bullen (who wrote Bullen 2013) and Peter Hajek (who wrote Hajek 2019) are among the authors of the Cochrane review, putting them in the position of assessing their own work.
Stanton Glantz blog 