FDA’s proposed guidance to industry on how to conduct product perception studies is a good start, but needs more specifics

My colleagues at the UCSF TCORS, in collaboration with Stanford, GSU and U of Iowa just submitted a public comment to FDA on its draft guidance on how tobacco companies should conduct perception studies of their products that are to be submitted to FDA as part of applications for approval to market these products or make reduced exposure or risk advertising claims. The general guidance the FDA is proposing follows best practice for such experiments in general, but needs more specifics on several issues to take into account unique aspects of tobacco and the tobacco companies’ behavior.

The full comment follows. A PDF is available here. The submission number on Regulations.gov is kiz-5uf9-042s.

FDA’s draft guidance on Principles for Designing and Conducting Tobacco Product Perception and Intention Studies appropriately highlights the importance of determining whether consumers understand the risks of new tobacco products and modified risk claims, but should provide more specific guidelines concerning youth perceptions, measuring intentions to use, addressing perceived benefits, considering effects on bystanders, assessing relapse and dual use, conducting qualitative studies, addressing null findings, and comparing proposed products to products currently commercially available

Docket No: FDA-2019-D-4188

Lauren K. Lempert, JD, MPH; Stanton A Glantz, PhD; Pamela M. Ling, MD, MPH;

Lucy Popova, PhD1; Benjamin Chaffee, DDS, PhD; Shannon Lea Watkins, PhD2;

Minji Kim, PhD; Bonnie Halpern-Felsher, PhD3

UCSF TCORS

1. Department of Health Policy & Behavioral Sciences, School of Public Health, Georgia State University

2. Department of Community & Behavioral Health, College of Public Health, University of Iowa

3. UCSF TCORS and Division of Adolescent Medicine, Department of Pediatrics, Stanford University

December 21, 2020

      FDA’s draft guidance on tobacco product perception and intention studies (TPPI) provided the industry with broad suggestions, rather than detailed guidelines, on how companies should design and conduct studies to strengthen their modified risk tobacco product applications (MRTPA), premarket tobacco product applications (PMTA), or substantial equivalence reports (SE). While we generally agree with the broad principles for TPPI studies that FDA outlined, there are eight areas where the guidance document should provide more specific guidance:

  1. TPPI studies should consider youth perceptions associated with the tobacco products as well as adult perceptions, although tobacco companies should not conduct studies on youth directly and should instead follow strict safeguards;
  2. TPPI studies should include measures of willingness and susceptibility to use tobacco;
  3. Studies should address perceived benefits as well as the harms of using a tobacco product;
  4. Studies should consider the actual and perceived effects of the product on bystanders;
  5. TPPI studies should assess whether the proposed product promotes relapse among former tobacco product users and whether it promotes dual- or poly-use;
  6. Qualitative studies should use rigorous data analysis methods;
  7. Studies should address null findings and exclude the possibility that null findings are due to poor quality measures or low statistical power; and
  8. Studies used to support PMTA and MRTP applications should compare the proposed product to other products currently on the market, not just conventional, combustible cigarettes.

We discuss these eight areas in greater detail in Section 3, starting on p. 3

  1. Submission and evaluation of tobacco product perception and intention studies regarding new tobacco products and modified risk products are essential for FDA to protect the public health

FDA issued a draft guidance for industry on how to design and conduct tobacco product perception and intention studies (TPPI Guidance)[1] that may be submitted to support modified risk tobacco product applications (MRTPA), premarket tobacco product applications (PMTA), or substantial equivalence reports (SE). These studies are used to assess individuals’ perceptions of tobacco products; understanding of tobacco product information including labeling, warnings, and modified risk information; and intentions to use tobacco products. TPPI studies are important because they can help FDA determine whether a manufacturer has demonstrated that a new tobacco product for which a PMTA has been submitted meets the “appropriate for the protection of the public health” (APPH) standard required by the Family Smoking Prevention and Tobacco Control Act (TCA) section 910(c) or that a modified risk or modified exposure for which a MRTPA has been submitted will “benefit the health of the population as a whole” under TCA section 911.

            FDA has reportedly received thousands of PMTAs for e-cigarettes and other new tobacco products by the September 9, 2020 submission deadline that should have contained TPPIs, and marketing orders issued to successful applicants may require them to submit TPPIs as part of their postmarket reports. Therefore, the appropriate design, conduct, and reporting of TPPI studies will be central to FDA’s decision-making and ability to protect the public health.

  • FDA’s draft guidance appropriately provides broad recommendations for designing and conducting perception and intention studies

FDA’s draft guidance addresses several issues for applicants to consider when designing and conducting studies, including:

  • developing study aims and hypotheses
  • designing quantitative and qualitative studies
  • modes of data collection
  • selecting and adapting measures
  • determining study outcomes
  • selecting and justifying study samples
  • analyzing study results

These guidelines are broad and provide a good summary of best practices for conducting the studies that are necessary for FDA to consider when making its determinations on PMTAs, MRTPAs, and SEs, and in evaluating postmarket reports. We support these broad recommendations, but as detailed below, recommend that more specific guidelines be provided in some areas.

  • FDA should provide more specific guidelines in certain areas

In some cases, FDA has provided more specific guidelines that are helpful. For example, FDA correctly specified that studies on consumer understanding of the claims made in the product’s labeling and marketing should measure to what extent consumers understand that the reduction of risk from a modified risk tobacco product is contingent on using the product exclusively (i.e., that they must switch completely to this product, rather than use this product along with one or more other products).  This issue was central, for example, to Philip Morris’s claims in its MRTPA that IQOS reduces consumers’ exposure to some harmful chemicals.  In previous public comments and publications,[2], [3], [4], [5], [6] we pointed out that Philip Morris’s consumer perceptions studies did not adequately measure consumers’ understanding of this essential factor.

In our previous public comments and publications cited above, we also pointed out that PMI’s data showed that consumers do not understand what it means to “switch completely” to IQOS and are likely to use IQOS concurrently with conventional cigarettes or other tobacco products (“dual use” or “poly use”), thereby reducing or eliminating the claimed benefits. PMI’s advertising and labeling do not adequately describe the conditions of use – namely, that to (allegedly) reduce their risk of tobacco-related diseases, consumers must use IQOS exclusively, and may not use it with any other tobacco product.

PMI’s studies did not, but should have, demonstrated that consumers understand these conditions of use. Further, the studies cited by PMI in support of their IQOS claims failed to include data on whether adolescents understood these claims and whether they would be interested in initiating tobacco with IQOS.[7] In an empirical study in which adolescents and young adults (mean age 19.3) were shown the IQOS proposed claims for reduced risk and exposure, 1 in 4 did not understand what “switch completely” meant.[8] 

FDA’s April 30, 2019 Marketing Order for IQOS requires Philip Morris to submit postmarket reports on an annual basis beginning April 30, 2020 including:

  • A summary of all formative consumer research studies conducted – whether by you, on your behalf, or at your direction – among any audiences, in the formation of new labeling, advertising, marketing, and/or promotional materials, including qualitative and quantitative research studies used to determine message effectiveness, consumer knowledge, attitudes, beliefs, intentions and behaviors toward using the products, and including the findings of these studies and copies of the stimuli used in testing.
  • A summary of all consumer evaluation research studies conducted – whether by you, on your behalf, or at your direction – among any audiences, to determine the effectiveness if labeling, advertising, marketing and/or promotional materials and any shifts in consumer knowledge, attitudes, beliefs, intentions, and behaviors toward using the products, and including the findings of these studies and copies of the stimuli used in testing.[9]

Similarly, FDA’s July 7, 2020 Modified Risk – Exposure Modification Order for IQOS requires Philip Morris to “conduct postmarket surveillance and studies in order to ‘determine the impact of the [MRTP exposure modification] order on consumer perception, behavior, and health, and to enable the [FDA] to review the accuracy of the determinations upon which the [MRTP exposure modification] order was based…’”[10]  In particular, FDA stated that in Philip Morris’s postmarket behavioral cohort study,

monitoring use of the [IQOS] products that are the subject of [the Exposure Modification] order in terms of uptake, dual use, and complete switching is required. In particular, your PMSS [postmarket surveillance and studies] must address the extent to which new MRTP users were never, former, or current smokers, or other tobacco product users before initiating the MRTPs and the extent to which new users of the MRTPs become exclusive IQOS users, dual users with combusted cigarettes or other tobacco products, or transition to combusted cigarette smoking over time.  These studies should be designed to observe behavior over a sufficient period of time to examine, for instance, the extent to which dual use of IQOS and combusted cigarettes is a transitional versus stable pattern of use.

 Philip Morris’s postmarket reports should also include studies that address these issues of consumer understanding. FDA should withdraw its reduced exposure order if these reports do not demonstrate that consumers, including youth, current cigarette smokers, and former smokers, understand that they must quit using conventional cigarettes altogether and must switch completely to IQOS to get the claimed benefits of reduced exposure. These studies should follow the specific recommendations for TPPIs outlined below in Section 3. Any future postmarket reports that are required as a condition of any future MRTP order should include similar studies that demonstrate whether, as actually marketed and used, consumers understand that they will not get the claimed benefits of the MRTP product unless they use that product exclusively. However, postmarket studies are not a substitute for required TPPI studies in PMTAs and MRTPAs.

While the broad principles for TPPI studies outlined in FDA’s guidance are good in general, the experience with PMI’s PMTA and MRTPA for IQOS highlight the need for more specific guidelines, changes, or additions in several areas due to the unique issues raised in tobacco product regulation. Our recommendations are outlined below.

  1. TPPI studies should specifically consider youth perceptions as well as adult perceptions, although tobacco companies should not conduct studies on youth directly and should instead follow strict safeguards

Although FDA recommends that applicants evaluate the potential impact of marketing a proposed product on “populations of interest” including users, former users, and non-users of tobacco products (TPPI Guidance p. 17, lines 663-678), it does not make clear that it is essential for TPPI studies to specifically evaluate perceptions and intentions for youth. This is a glaring omission. TPPI studies regarding youth perceptions are particularly important given the fact that 90% of long-term smokers began smoking as adolescents.[11]  Moreover, youth are most likely to initiate tobacco use with newer tobacco products such as e-cigarettes.  The literature clearly shows that adolescents are the most likely to initiate with and use e-cigarettes.  National data from 2019 show that 27.5% of high school students and 10.5% of middle school students reported past 30-day use of e-cigarettes.[12] Among young adults (18-24 years old), 7.6% reported past 30-day e-cigarette use.[13] In contrast, among adults only about 4.2% of those 25-44 and 2.1% of adults 45-64 reported past 30-day e-cigarette use.[14]  Youth tobacco use is especially concerning given that the brain continues to develop and change until the mid-20s, making adolescents and young adults especially sensitive to nicotine addiction.[15], [16], [17],[18]  

To determine the population level impact of a new or modified risk tobacco product, studies should be conducted that assess the potential impact of each new product on youth initiation and progression to established use of any tobacco product (not just the proposed new product), including comprehensive evaluations about how the flavors, nicotine delivery, and product marketing and labeling influence adolescents’ perceptions of both health harms and perceived benefits (e.g., perceived reduction in stress; looking cool or popular) the tobacco product may have on them and others,[19] the product appeal, addictive potential, intentions to use, actual use, product switching, and potential product switching and poly-use among youth.

To be clear, while our point is that studies on youth should be included in all PMTAs and MRTPs (as well as postmarket reviews), we are not recommending that the tobacco industry conduct TPPI studies on youth directly. The tobacco industry has a long history of manipulating research studies including the design, samples and sampling, analyses, and interpretation to be in favor of the industry, and to attract young and new users to their products.[20], [21], [22], [23] As such, we recommend the following: (1) all TPPI studies include perception studies on young adults, (2) PMTAs and MRTPAs  discuss the separate, independent published literature on youth on the same or similar products, and (3) that PMTAs and MRTPAs discuss the implications of studies of young adults for predicting youth outcomes. 

While it is essential that the FDA understand a proposed product’s impact on adolescents’ perceptions, intentions, and willingness to use, the tobacco industry’s long history of manipulating research cannot be understated. Furthermore, given the tobacco industry’s well documented history of marketing to youth,[24], [25]  including use of youth “smoking prevention” research programs as cover for these activities,[26] we do not endorse any process that would allow or encourage industry research directly on youth. In particular, industry research conducted on youth to purportedly demonstrate perceptions to FDA would likely be used by the industry for its own product development and marketing purposes.

 Studies on youth perceptions can be and have been conducted ethically by academic researchers independent of the tobacco companies, such as survey studies in which youth are provided with a picture of the new product and/or a description of the product, and youth are asked about their perceptions of the product or asked to interpret marketing statements that are proposed by the company. For example, in a recent study of Philip Morris International’s proposed marketing for IQOS, youth were randomly assigned to see either a “reduced exposure,” “reduced risk,” or neither claim. Perceptions of IQOS-related health risks and general harm and understanding of the term “switching completely” as used in PMI’s proposed claims were compared. Results showed that youth exposed to “reduced risk” claims perceived lower general harm than the control group, and that 30% of the youth didn’t understand the term “switching completely.”[27] In a study of California youth and young adults (mean age 17.5, SD = 1.7), participants were asked to indicate whether eight different ads for flavored e-cigarette products, randomly displayed, target someone younger than them, their age, someone a little older, or someone much older like their parents. Participants felt the ads were for someone just a little older than them (age 18 – 26; not for someone much older). More than half of participants felt ads for cherry, vanilla cupcake, caramel, and smoothie flavors were for someone their age. Ads were also seen as targeting an audience younger than them.[28] Finally, survey-based studies and qualitative studies have assessed youth’s understanding and perceptions of different tobacco products, showing that youth often harbor misperceptions about these new products that could encourage use.  

A 2012 Institute of Medicine Report and a July 2020 peer-reviewed published commentary[29] on the importance of including youth in studies related to PMTAs, MRTPs, and SEs suggest ways in which the FDA can conduct their own research or fund and oversee external research on the tobacco products submitted through the MRTPA or PMTA process to inform its decision making.  We reproduce verbatim below the specific recommendations made in the July 2020 paper, which was based on the literature and the FDA-commissioned Institute of Medicine/National Academies of Science report on Scientific Standards for Studies on Modified Risk Tobacco Products.[30]

  1. Empirical evidence related to harm perceptions, product appeal, and the addictive potential among youth for any proposed product or claim must be included in every application. Since the TCA places the evidentiary burden on manufacturers, it is likely that some of this research will be funded by the tobacco industry. Given the history of tobacco industry manipulation of research,[31] FDA must establish the following specific safeguards to ensure that the evidence is objective, reliable, and protected from industry influence:
    1. All studies must receive Internal Review Board Approval to ensure that the research is ethical and protects human subjects.
    1. All studies should be conducted by a third-party, independent group of investigators. FDA must provide guidelines for study criteria, the research questions to be addressed, the independent groups conducting the research, and the quality checks needed. FDA must also set clear rules on data transparency so that the industry cannot prevent the investigators from presenting the data to FDA or the public. FDA should also periodically evaluate the independence of the studies and the respective third-party research groups to assess the possibility of industry influence.
    1. All research protocols must be listed on http://www.clinicaltrials.gov; be accessible to the public; and meet minimum standards for designing, conducting, and reporting results for studies. All study procedures must be stated clearly to be completely transparent and reproducible.
    1. An independent review committee (IRC) with rotating membership, with no financial ties to the tobacco industry, must be appointed by the FDA to review and approve research protocols. Higher risk protocols should also include an independent Data Safety and Monitoring Board to monitor ongoing progress.
    1. Studies must examine specific risk perceptions related to short- and long-term health outcomes, risk of addiction, and perceptions of the new product compared to other products already on the market (e.g., including but not limited to cigarettes).
    1. Studies must carefully assess each specific claim, proposed marketing, and promotional efforts including color and style of the product packaging.
    1. Studies must include examination and documentation of the impact of constituents among youth users. While such exposure studies are critically important, they need to follow federal and local laws, and as such may be difficult to conduct among younger youth. In such instances, studies conducted among young adults could be presented and implications of the findings to younger youth should be discussed.
    1. Studies should include nationally representative youth samples that reflect sufficient sample size with variation in socioeconomic status, race/ethnicity, sex, geographic location, and use patterns. Findings from different age categories should not be inferred to youth, except as discussed above.
    1. Proposed studies must follow the guidelines proposed by NIDA for substance use research involving children and adolescents, and if appropriate for exposure studies in human subjects.[32], [33]
  2. All applications must include a review of existing comparative studies of similar products, including research on adolescent perceptions as they relate to intentions to use and actual use patterns. This review does not replace the requirement of submitting evidence specific to the products and claims being considered.
  3. Authorization of any new tobacco products must be based on evidence specific to youth in the U.S. Evidence from other countries can be considered but should not serve as the primary source of information.

Consistent with the study guidelines and criteria recommended in 1.b. above, the process should include a commitment by PMTA and MRTP applicants to use the knowledge obtained in the studies to affirmatively modify the design of their proposed product, the product’s claims, and its marketing in a way that repels or discourages rather than attracts or encourages youth to use or purchase the product and that will not lead to youth initiation of any tobacco product. All PMTAs and MRTPAs should describe in detail how the product design, claims, and marketing materials used or will use the acquired information. As a condition of PMTA and MRTP authorizations, FDA should require that postmarket reports demonstrate with specific evidence how the product designs and marketing used the information from the studies to discourage, rather than attract, youth use. FDA should withdraw any marketing, modified risk, or modified exposure authorization for any product that fails to meet these requirements.

While the Tobacco Control Act clearly puts the burden on the applicant to demonstrate with scientific evidence that its proposed product is “appropriate for the protection of the public health,” FDA can also conduct its own research on youth perceptions to help inform its decision making on PMTA and MRTP applications. Additionally, FDA can fund external research on tobacco products and/or on classes or categories of products, rather than on specific products, to protect confidential and/or proprietary industry information.

  • Because intentions are not a proxy for actual behavior, especially for adolescents, TPPI studies should include measures of willingness and susceptibility to use tobacco

According to decision-making theories such as the Social Cognitive Theory,[34] the Health Belief Model,[35] the Theory of Reasoned Action,[36] and the Theory of Planned Behavior,[37] people’s behaviors are largely shaped by their intentions to engage in that behavior.  These intentions are, in turn, shaped by their perceptions of behavior-related risks and perceived benefits. While these theories have some merit, they are largely relying on cognitive processes, whereby one is expected to have a deliberate, planned decision to or not to engage in a behavior.  In these cases, intentions are more likely to lead to or predict actual behavior.  However, as discussed in detail below, studies show that these cognitive models do not accurately or fully predict how some people and in particular adolescents decide whether or not to engage in a behavior, including tobacco use. 

  Research demonstrates that decision-making does not always involve a deliberate, analytic process. Instead, many decisions, including adolescents’ decisions to use tobacco, are based more on heuristic, reactive, and affective processes.[38] While adolescents may not have an active plan in mind to smoke, they often find themselves in situations in which they would consider smoking even though they were originally committed to avoiding it. Willingness to smoke is shaped by perceptions, including perceived peer norms and peer acceptance of smoking as well as images associated with smoking. For example, adolescents are less likely to smoke if they hold negative images that smokers are dirty, wrinkled, and have yellow teeth. In contrast, adolescents who are exposed to positive images of smokers are more likely to view smoking favorably and therefore try smoking.[39] As such, willingness is a better predictor of tobacco use than intentions and should be used in studies examining whether and why anyone, and in particular an adolescent, would use any tobacco product.[40],[41]

Therefore, the FDA should recommend inclusion of studies in PMTA and MRTP applications that not only address youth intentions, but also youth willingness and susceptibility to use the proposed tobacco product.  Having said this, to reiterate, tobacco companies should not conduct studies on youth directly, and any studies submitted by tobacco companies must follow the strict safeguards outlined above. Instead, as noted above, the companies should include in their PMTA and MRPT applications studies from the existing literature on youth’s perceptions of different tobacco products.  

  • Studies should address perceived benefits of using a tobacco product as well as perceptions of risks

Decision-making theories and empirical studies argue that perceptions of both tobacco-related risks and perceptions of benefits are critical influences on intentions to use, willingness to use, and actual tobacco use.[42],[43],[44]    For example, studies have shown that one’s decision to engage in a risky behavior, including tobacco use, is influenced by both perceptions of risks and perceptions of benefits, and that both factors independently contribute to predicting tobacco use intentions, willingness, and actual behavior. Further, studies of adolescents’ perceived benefits in addition to risks can explain why youth use tobacco despite knowing some of the risks.  Studies show that perceptions of benefits can be associated with actual tobacco use over and above perceptions of risks.[45]

Therefore, FDA should recommend that studies be conducted on not just perceptions of risks associated with the product, but also on perceptions of benefits. Studies of youth (following the guidelines we recommend above) and adults should measure the perceived benefits of the product, including: perceived benefits among users, never users, and former users; perceived benefits of the proposed tobacco product compared to other products on the market; and perceived benefits of switching from one tobacco product to another.

  • FDA should require TPPI studies to include measures of perceived risk for bystanders and non-users resulting from secondhand smoke/aerosol exposure

             Perceptions of risk to self from using a tobacco product are different from perceptions of risk to bystanders and non-users who might be exposed to secondhand smoke/aerosol and sidestream emissions. Advertisements about tobacco products being cleaner and having “no smoke” might mislead people into believing that aerosol from e-cigarettes or heated tobacco products is harmless for others.[46] There is substantial literature showing that concern over harm to others (e.g., from secondhand smoke exposure) is a more effective motivator for cessation and other behavior change than concerns over harm to oneself.[47], [48], [49], [50], [51] FDA should require TPPI studies to report on the measures of perceived harm of the products on bystanders in addition to perceptions of risk to oneself.

  • FDA should broaden its guidance to stress that TPPI studies should assess whether the proposed product promotes relapse among former tobacco users and whether it promotes dual or poly use

The TPPI Guidance (TPPI Guidance p. 6, lines 189-193) appropriately recommends that studies should assess whether smokers would likely start using the proposed product and stop smoking cigarettes, and whether nonsmokers would likely initiate use of the product. However, the Guidance does not make clear that TPPI studies should also assess whether the proposed product is likely to cause relapse among former tobacco users and whether the proposed product is likely to promote dual or poly use. While the FDA appropriately recommends (TPPI Guidance p. 14, lines 527-529) that studies should investigate consumers’ perceptions of the health risks of dual use, it does not clearly define what constitutes “dual use” or the need for users to switch completely to the proposed new product to obtain the claimed health benefits and it does not explicitly recommend an examination of dual use.

The 2019 National Youth Tobacco Survey found that about 1/3 of current middle school and high school tobacco users are dual- or poly-users, [52] and a November 2020 CDC analysis of the 2019 National Health Interview Survey (NHIS) data found that about 38% of adults are dual- or poly- users.[53]  Nevertheless, FDA seems to allow the applicant to decide whether exclusive or dual use of its proposed product is likely (TPPI Guidance p. 15, lines 567-572, 594, 619-621). FDA should delete the sentence (beginning at line 570), “Alternatively, the behavioral intentions may be assessed to determine whether a current user is likely to entirely replace their current product with another product – for instance, entirely replacing combusted cigarettes with a non-combusted product” and the parenthetical phrase “(e.g., exclusive use)” (at line 594). Instead, FDA should specify that studies should explicitly address dual use and applicants should measure whether consumers understand that dual use is likely to result in a net increase in risk over use of one product alone.[54], [55]

Indeed, a November 2020 CDC report on adult tobacco use showed that while the rate of adult smoking has remained constant at 14% since 2017,[56] the rate of adults who use e-cigarettes increased during the same time period, suggesting that e-cigarette use is not effective in helping smokers quit.[57]  Other population-based studies have similarly found no evidence of e-cigarette effectiveness at the population level; for example, a longitudinal study using the Population Assessment of Tobacco and Health (PATH) Study data found that e-cigarettes were not associated with short- or middle-term cessation among smokers who made a quit attempt.[58] This follows the January 2020 Surgeon General’s report on smoking cessation that found, “There is presently inadequate evidence to conclude that e-cigarettes, in general, increase smoking cessation.”[59]

The problem of not appropriately addressing dual use is highlighted in a study of the PMTA for IQOS.[60] In the IQOS PMTA, Philip Morris defined “switching completely” to IQOS as up to 30% use of conventional cigarettes concurrently with IQOS. FDA acknowledged in its PMTA decision for IQOS[61] that Philip Morris failed to evaluate dual use in its discussions of individual or population level health impacts.  FDA expressed concerns about the effects of dual use of IQOS (compared with complete switching) on long-term reduction of exposures to harmful and potentially harmful constituents (HPHCs) and concluded that the health benefits of reducing cigarette consumption instead of quitting completely remained unclear. Nevertheless, FDA wrongly authorized the marketing of IQOS in the US.[62] It is essential that TPPI studies investigate how products are actually used and should not rely on the industry’s unorthodox definitions of “dual use” or “complete switching.” FDA should clarify that any examination of “dual use” in TPPIs means at least 5% use of other tobacco products, and anything more than 5% dual use does not constitute “completely switching” to the proposed product.

  • Qualitative studies should use rigorous data analysis methods to minimize bias

Like quantitative studies, qualitative analyses are subject to the biases of those conducting the research during data collection, in the analysis, and in interpretation of the data.  While the FDA draft guidance correctly lays out basic standards for qualitative data collection, the FDA should recommend that qualitative studies take measures to minimize bias in qualitative data analysis,[63] including but not limited to:

  • documentation of data collection and analysis processes
  • use of coding guides, and making these guides available for independent review
  • documentation of measures to check intercoder reliability and consistency of interpretation
  • authenticate conclusions taking into account the credibility of respondents, reflexivity and researcher interactions, and triangulation between different data sources

Further, qualitative transcripts should be made available for review by independent scientists to address bias in data collection and interpretation. Finally, qualitative research should be presented in accord with scientific standards of rigor, such as utilizing CASP (Critical Appraisal Skills Programme) checklists to ensure data are analyzed and presented rigorously.  The CASP Qualitative Studies Checklist[64] is a good tool to follow; CASP also provides checklists for quantitative studies.  Several other scholars[65], [66], [67], [68] have provided checklists and criteria that should be met to support rigorous qualitative methods.

  • FDA should expand its discussion of the importance of addressing null findings and excluding the possibility that null findings are due to poor quality measures or low statistical power

FDA appropriately discusses the importance of addressing null findings and excluding the possibility that such findings are due to poor quality measures or low statistical power (TPPI Guidance p. 6, lines 213-214; p. 17, lines 696, 710-722). This issue is important and should be given even stronger attention because in perception and intention studies there are many cases in which the applicant may want null findings (e.g., to show that there is insufficient evidence to demonstrate that youth misperceive the health risks of a product). In contrast, in studies seeking to show the efficacy of a new drug, the applicant is strongly motivated to achieve positive results. The FDA should stress that TPPIs should contain a specific statement about the minimum detectable effect size that the applicant is powering the study to detect together with a strong justification for that effect size.

The conventional assumption that 80% is an acceptable power is probably too low because null findings are likely to be important. For example, an applicant may wish to argue that the marketing materials for a new product did not appeal to youth or that a new product does not increase relapse to tobacco use among former users.  Therefore, in parallel with the standard practice for controlling type I error for positive findings, a power of 95% to support null findings would be comparable to the level widely used for reaching positive conclusions.

FDA correctly urges applicants to provide scientific rationale for how they dealt with Type I error with multiple comparisons. (TPPI Guidance p. 20, line 772). The biomarkers of potential harm studies submitted in the IQOS PMTA were rife with these kinds of errors, and an independent analysis of these studies concluded that the data submitted by Philip Morris failed to show consistently lower risks of harm in humans using IQOS compared with conventional cigarettes because there were not statistically significant differences between IQOS and conventional cigarette users for 23 of 24 of the biomarkers of potential harm studied.[69]

Moreover, PMI used an arcane, little-used statistical method (the “Hailperin-Rüger method”) that has many problems. As we noted in a December 2018 public comment,[70] this method is:

…overly cautious to require that all observed changes be statistically significant in order to confirm that a therapy works and that if some lesser number of the variables change significantly, that should be good enough for a global test. The number of significant changes is specified in advance and the probability of a chance finding is adjusted.

PMI decided that if 5 of the 8 biomarkers (6 clinical risk and 2 exposure) changed in the direction of less risk, that would be enough to conclude that IQOS was less risky than conventional cigarettes. They do not provide a clear explanation of why they used 5, other than it was “more than half.”

PMI justified using Hailperin-Rüger because “the probability of finding five significant tests (p<0.05) by chance alone is extremely low (0.006%).” This is a misleading statement because this low probability would only be the case a chance finding if none of the five variables actually changed. The probabilities are much higher when there are real changes.

So, in the new study, PMI went from considering changes in 24 clinical risk biomarkers in the original study to 8 in the new study to only requiring 5 to be statistically significant. That is a pretty major drop in the level of evidence PMI now suggests is sufficient to demonstrate that IQOS is less risky than cigarettes.

In the new study 5 of the changes were statistically significant, so PMI concluded that, overall, IQOS was better. Had they picked 6 in their plan, the overall results would not have been significant, even under the Hailperin-Rüger method’s relaxed standards.

  • FDA should make clear that when designing and conducting studies for PMTAs and MRTPAs, applicants should compare the proposed product to other products that are currently on the market and not just conventional cigarettes.

FDA’s Guidance on Applications for Premarket Review of New Tobacco Products[71] and Guidance on Premarket Tobacco Product Applications for Electronic Nicotine Delivery Systems[72] make clear that applicants should compare the new product with other tobacco products currently on the market. However, FDA’s definition of “comparison product” (TPPI Guidance, p. 3, lines 87-91) and discussion of addressing perceptions of health risks of dual use of the product and the comparison product (TPPI Guidance, p. 14, lines 527-529) do not clarify that an applicant should compare the proposed new or modified risk/exposure product to a wide range of different types of tobacco and nicotine products that are currently commercially available, including commercially marketed e-cigarettes (including pod-based, disposables, tanks, mods, etc.), heated tobacco products, smokeless, and any other currently commercially product, and not merely to conventional cigarettes.

There are many reasons why TPPI studies need to compare perceptions and intentions to use the new proposed product not just to conventional cigarettes.  For example, as noted above, many adults and youth are using multiple tobacco products at once. Further, youth have negative perceptions and attitudes towards cigarettes[73] and are less likely to use cigarettes than other tobacco products. Thus, comparing any new product to cigarettes in studies of youth is inaccurate, insufficient, and less relevant than comparing that new product to more popular products such as e-cigarettes and cigars.

Finally, there are differences in tobacco product use by sex, race/ethnicity, geography, context of use, and so on.[74]  Comparing new products only to cigarettes fails to recognize products that are more popular among some marginalized groups such as cigars among young people of color or smokeless tobacco among rural users. Requiring TPPI studies to compare new products to an array of existing products would be more inclusive of the perceptions and experiences of all tobacco users and have greater potential to avoid exacerbating existing tobacco-related disparities. 

As such, when evaluating new product applications, FDA must consider whether marketing the proposed product would reduce the overall burden of tobacco product use by considering actual use patterns of not just conventional cigarettes, but other tobacco products as well. This means that the manufacturer must compare the proposed product’s potential harms with those of tobacco products that were on the market at the time FDA makes its PMTA or MRTPA evaluation. Because Philip Morris’s PMTA for IQOS compared the health risks of IQOS only to conventional cigarettes and not to e-cigarettes or other products currently on the market, FDA’s evaluation of and marketing authorization for IQOS was flawed.[75] FDA’s TPPI Guidance should specify that TPPI studies should compare the subject product to all other products that are currently on the market and not only to conventional cigarettes.

  • Conclusion

FDA’s draft guidance on designing and conducting tobacco product perception and intention studies appropriately recognizes that these studies are essential to FDA’s review and determination of MRTP and PMTA applications and SE reports. Because it is likely that FDA will be reviewing thousands of PMTAs for e-cigarettes and other newly deemed tobacco products in the coming year, this guidance is timely and important.  While we generally agree with the broad principles for TPPI studies that FDA outlined, as summarized at the beginning of this comment, there are eight areas where the guidance should be modified because more specificity is needed:

  1. TPPI studies must consider youth perceptions associated with tobacco products as well as adult perceptions, although tobacco companies should not conduct studies on youth directly and should instead follow strict safeguards;
  2. Because intentions are not a proxy for actually behavior, especially for adolescents, TPPI studies should include measures of willingness and susceptibility to use tobacco;
  3. Studies should address perceptions of the benefits of using a product as well as perceptions of risks;
  4. Studies should consider the actual and perceived effects of the proposed product on bystanders;
  5. TPPI studies should assess whether the proposed product promotes relapse among former users and whether it promotes dual- or poly-use;
  6. Qualitative studies should use rigorous data analysis methods;
  7. Studies should address null findings and should be based on high quality measures in studies with high statistical power; and
  8. Studies used to support PMTA and MRTP applications should compare the proposed product to other products currently on the market, not just conventional cigarettes.

Appendix: How TPPI studies will be used by FDA

  1. TPPI studies in premarket tobacco product applications

In its June 2019 guidance on Premarket Tobacco Product Applications for Electronic Nicotine Delivery Systems,[76] FDA recommends that PMTAs provide human health impact information that “adequately characterizes the potential impact of the new tobacco product on the health of both users and nonusers of tobacco products” to support a finding that permitting the marketing of the new tobacco product would be appropriate for the protection of the public health (APPH). To fully assess these health effects, FDA recommends that PMTAs include consumer perception evaluations that “address how consumers perceive product harms and include consideration of packaging and labeling.  These evaluations should also address interest in and intentions to use the product, including among populations of non-users of tobacco products (e.g., vulnerable populations such as youth and young adults).” Additionally, FDA recommends that applicants evaluate perceptions of the product, “both absolute and in comparison to other categories of tobacco products and to quitting all tobacco use.  This evaluation should include the use intentions among current ENDS users, nonusers, and other tobacco product users, as well as reasons for use…” These studies should include adequate representation of diverse participants, including groups disproportionately affected by tobacco related harms

  • TPPI studies in modified risk tobacco product applications

Similarly, manufacturers must demonstrate that a product for which a MRTPA has been submitted will “benefit the health of the population as a whole” considering both users and nonusers of tobacco products, and TPPI studies can help FDA determine whether a manufacturer has met this burden. TPPI studies may demonstrate that perceptions of less risk do not result in nontobacco users (including youth) initiating tobacco use, in existing tobacco users who would otherwise quit using tobacco products switching to the new product, or in overall increased use of tobacco products. Importantly, consumer perception testing is required under TCA section 911(g)(2)(B)(iii) for MRTP applicants seeking exposure modification orders to demonstrate that the proposed labeling and marketing of the product does not mislead consumers into believing that the product is less harmful or that the product presents less of a risk of a disease than one or more other commercially marketed tobacco products. (See also March 2012 guidance on Modified Risk Tobacco Product Applications.[77])

The Institute of Medicine’s report on Scientific Standards for Studies on Modified Risk Tobacco Products[78] (IOM report) determined that studies of risk perceptions about the MRTP are necessary to support MRTP marketing authorization because individuals’ perceptions play a key role in behavioral choices. In particular, these studies are “important to identify consumers’ perceptions of disease risk, likelihood of addiction, likelihood of reducing or increasing others’ exposure to potentially hazardous compounds, and perceptions of risk compared to other products already on the market,” as well as to assess intentions of using the product. Further, the IOM report said it is essential that the industry “demonstrates through rigorous testing that people correctly understand and interpret the risks.” The IOM report found that studies evaluating risk perceptions and risk communication should be performed both before and after the marketing of an MRTP. The premarket studies are important “to determine consumers’ ability to accurately understand messages that communicate information about the risks, benefits, and conditions of using an MRTP compared to existing tobacco products” and to “test how these messages influence consumers’ perceptions of the risks, benefits, and likelihood of addiction related to an MRTP.” Further, they are important to evaluate consumer understanding and to compare consumer perceptions of an MRTP to other products on the market. The IOM report found that postmarket studies are “vital to continue monitoring consumer perceptions and behavior” related to the MRTP product.

  • TPPI studies in postmarket reports

Additionally, FDA has the authority under TCA sections 910(f), 911(g)(2)(c)(ii) and 911(i) to require successful PMTA or MRTP applicants to submit postmarket reports to help FDA determine whether the PMTA products continue to be appropriate for the protection of the public health or to determine the impact of the MRTP order on consumer perceptions, behavior and health. FDA can withdraw any PMTA or MRTP authorization on the basis of these reports for many reasons including if the reports show that the product is no longer APPH; nonusers (including youth) are initiating with the new product; current users are not quitting, but instead are dual using; consumers do not understand the labeling; the labeling or advertising is misleading; or consumers do not understand the risk claims.

Both FDA’s April 30, 2019 marketing order for Philip Morris’s IQOS[79] and its December 17, 2019 marketing order for 22nd Century’s Moonlight and Moonlight Menthol very low nicotine cigarettes,[80] require the applicants to submit annual postmarket reports including:

  • A summary of how the new product continues to be appropriate for the protection of the public health;
  • A summary of all formative consumer research studies conducted… in the formation of new labeling, advertising, marketing, and/or promotional materials, including qualitative and quantitative research studies used to determine message effectiveness, consumer knowledge, attitudes, beliefs, intentions and behaviors toward using the products…; and
  • A summary of all consumer evaluation research studies conducted… to determine the effectiveness of labeling, advertising, marketing and/or promotional materials and any shifts in consumer knowledge, attitudes, beliefs, intentions, and behaviors toward using the products…

            As a condition of its July 7, 2020 exposure modification MRTP order for the IQOS system and three flavors of Heatsticks,[81] FDA requires Philip Morris to conduct postmarket surveillance and studies to determine the order’s impact on:

  • MRTP use behavior, including impact on never, former, and current smokers and whether users become dual users
  • youth awareness and use
  • consumers’ understanding of the relative harms of the product
  • whether users understand that they much switch completely to IQOS to obtain any of the claimed benefits

[1] FDA. Tobacco Products: Principles for Designing and Conducting Tobacco Product Perception and Intention Studies, Draft Guidance (October 2020). Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/tobacco-products-principles-designing-and-conducting-tobacco-product-perception-and-intention

[2] Lempert LK, Kim M, Chaffee B, et al. FDA should not authorize Philip Morris International to market IQOS with claims of reduced risk or reduced exposure. Docket No. FDA-2017-D-3001, February 23, 2020. Available: https://tobacco.ucsf.edu/fda-should-not-authorize-philip-morris-international-market-iqos-claims-reduced-risk-or-reduced-exposure

[3] Lempert LK, Popova L, Halpern-Felsher B, et al. Because PMI has not demonstrated that IQOS is associated with lower risks, FDA should not permit modified exposure claims, because such claims are likely to be misunderstood as modified risk claims. Docket No. FDA-2017-D-3001, December 11, 2017. Available: https://tobacco.ucsf.edu/fda-should-not-permit-modified-exposure-claims-iqos-because-they-are-likely-be-misunderstood-modified-risk-claims

[4] Popova L, Lempert LK, Glantz SA. Light and mild redux: heated tobacco products’ reduced exposure claims are likely to be misunderstood as reduced risk claims. Tobacco Control 2018;27:s87-s95.

[5] Kim, M., Watkins, S. L., Koester, K. A., Mock, J., Kim, H. C., Olson, S., … & Ling, P. M. (2020). Unboxed: US Young Adult Tobacco Users’ Responses to a New Heated Tobacco Product. International Journal of Environmental Research and Public Health17(21), 8108.

[6] McKelvey, K., Popova, L., Kim, M., Chaffee, B., Vijayaraghavan, M., Ling, P., Halpern-Felsher, B. Heated tobacco products likely appeal to adolescents and young adults.  Tobacco Control. 2018 Nov;27(Suppl 1):s41-s47. PMID: 30352843.

[7] Halpern-Felsher B, McKelvey K, Kim M, et al. PMI’s MRTP Application for IQOS Does Not Consider IQOS’s Appeal to Youth or Adolescents, or the Likelihood that Youth and Adolescents will Initiate Tobacco Use with IQOS or Use IQOS with Other Tobacco Products. Docket No. FDA-2017-D-3001, December 7, 2017. Available: https://tobacco.ucsf.edu/pmi%E2%80%99s-mrtp-application-iqos-does-not-consider-iqos%E2%80%99s-appeal-youth-or-adolescents

[8]McKelvey, K., Baiocchi, M., Halpern-Felsher, B. PMI’s heated tobacco products marketing claims of reduced risk and reduced exposure may entice youth to try and continue using these products. Tobacco Control. Tobacco Control Published Online First: 06 February 2020. doi:10.1136/tobaccocontrol-2019-055318.

[9] FDA. Marketing Order, FDA Submission Tracking Numbers (STNs): PM0000424-PM0000426, PM0000479, April 30, 2019. Available: https://www.fda.gov/media/124248/download&nbsp;

[10] FDA. Modified Risk Orders – Exposure Modification, FDA Submission Tracking Numbers: MR0000059‐MR000061, MR0000133, July 7, 2020. Available: https://www.fda.gov/media/139797/download

[11] U.S. Department of Health and Human Services. National Survey on Drug Use and Health. Rockville, MD: Substance Abuse and Mental Health Services Administration (SAMHSA); 2014

[12] Cullen KA, Gentzke AS, Sawdey MD, et al. e-Cigarette Use Among Youth in the United States, 2019. JAMA. 2019. doi:10.1001/jama.2019.18387

[13] Villarroel MA, Cha, A.E., Vahratian, A. Electronic Cigarette Use Among U.S. Adults, 2018: NCHS Data Brief No.365. National Center for Health Statistics, Centers for Disease Control and Prevention, U.S. Department of Health and Human Services. https://stacks.cdc.gov/view/cdc/87918. Published 2020. Accessed 24 May 2020, 2020.

[14] https://truthinitiative.org/research-resources/emerging-tobacco-products/e-cigarettes-facts-stats-and-regulations

[15] Kim, M., Popova, L., Halpern-Felsher, B., Ling, PM.  Effects of e-cigarette advertisements on adolescents’ perceptions of cigarettes. Health Communication. 2017 Dec 13:1-8. PMID: 29236550

[16] Kim, M., Ling, PM., Ramamurthi, D., Halpern-Felsher, BL. Youth’s perceptions of e-cigarette advertisements with cessation claims. Tobacco Regulation Science. 2019 July;5(2):94-104. PMID: 31840040

[17] U.S. Department of Health and Human Services. E-Cigarette Use Among Youth and Young Adults: A Report of the Surgeon General. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2016

[18] Yuan M, Cross SJ, Loughlin SE, et al. Nicotine and the adolescent brain. J Physiol 2015;593:3397-3412.

[19] Song, A. V., Morrell, H. E. R., Cornell, J. L., Ramos, M. E., Biehl, M., Kropp, R. Y., & Halpern-Felsher, B. L. (2009). Perceptions of Smoking-Related Risks and Benefits as Predictors of Adolescent Smoking Initiation. American Journal of Public Health, 99(3), 487–492. https://doi.org/10.2105/AJPH.2008.137679

[20] Bero L. A. (2005). Tobacco industry manipulation of research. Public Health Reports (Washington, D.C.: 1974), 120(2), 200–208. https://doi.org/10.1177/003335490512000215

[21] Smith, E. A., & McDaniel, P. A. (2016). “The Policy Dystopia Model”: Implications for Health Advocates and Democratic Governance. PLoS medicine13(9), e1002126. https://doi.org/10.1371/journal.pmed.1002126

[22] Barnes DE, Bero LA. Industry-funded research and conflict of interest: an analysis of research sponsored by the tobacco industry through the Center for Indoor Air Research. J Health Polit Policy Law. 1996;21(3):515–42.

[23] United States v. Philip Morris, 449 F. Supp. 2d 1 (D.D.C. 2006) 

[24] Cummings KM, Morley CP, Horan JK, Steger C, Leavell NR. Marketing to America’s youth: evidence from corporate documents. Tob Control. 2002 Mar;11 Suppl 1(Suppl 1):I5-17. doi: 10.1136/tc.11.suppl_1.i5. PMID: 11893810; PMCID: PMC1766057.

[25] Perry CL. The tobacco industry and underage youth smoking: tobacco industry documents from the Minnesota litigation. Arch Pediatr Adolesc Med. 1999 Sep;153(9):935-41. doi: 10.1001/archpedi.153.9.935. PMID: 10482208.

[26] Landman A, Ling PM, Glantz SA. Tobacco industry youth smoking prevention programs: protecting the industry and hurting tobacco control. Am J Public Health. 2002 Jun;92(6):917-30. doi: 10.2105/ajph.92.6.917. PMID: 12036777; PMCID: PMC1447482.

[27] McKelvey, K., Baiocchi, M., Halpern-Felsher, B. PMI’s heated tobacco products marketing claims of reduced risk and reduced exposure may entice youth to try and continue using these products. Tobacco Control. Tobacco Control Published Online First: 06 February 2020. doi:10.1136/tobaccocontrol-2019-055318.

[28] McKelvey, K., Baiocchi, M., Ramamurthi, D., McLaughlin, S., Halpern-Felsher, B. Youth say ads for flavored e-liquids are for them. Addictive Behaviors. 2019 Apr;91:164-170. PMID:30314868

[29] Halpern-Felsher B, Henigan D, Riordan M, Boonn A, Perks SN, Krishnan-Sarin S, Vallone D. The Importance of Including Youth Research in Premarket Tobacco Product and Modified Risk Tobacco Product Applications to the Food and Drug Administration. J Adolesc Health. 2020 Sep;67(3):331-333. doi: 10.1016/j.jadohealth.2020.06.020. Epub 2020 Jul 14. PMID: 32674965

[30] Institute of Medicine 2012. Scientific Standards for Studies on Modified Risk Tobacco Products. Washington, DC: The National Academies Press. https://doi.org/10.17226/13294. Available: https://www.nap.edu/download/13294

[31] Institute of Medicine 2012. Scientific Standards for Studies on Modified Risk Tobacco Products. Washington, DC: The National Academies Press. https://doi.org/10.17226/13294. Available: https://www.nap.edu/download/13294

[32] National Institute on Drug Abuse. NACDA guidelines for substance abuse research involving children and adolescents. Available at: https://archives. drugabuse.gov/advisory-council-minutes/nacda-guidelines-substance-abuse- research-involving-children-adolescents.

[33] National Institute on Drug Abuse. NACDA guidelines for administration of drugs to human subjects. Available at: https://www.drugabuse.gov/research/ clinical-research/nacda-guidelines-administration-drugs-to-human-subjects.

[34] Bandura, A. (1994). Social cognitive theory and exercise of control over HIV infection. In R.J. DiClemente & J.L. Peterson (Eds.), Preventing AIDS: Theories and methods of behavioral interventions. New York: Plenum Press

[35] Rosenstock, I.M. (1974). Historical origins of the health belief model. In M.H. Becker (Ed.), The Health Belief Model and Personal Health Behavior (pp. 1-8). Thorofare, NJ: Charles B. Sclack.

[36] Fishbein M, Ajzen I. Belief, attitude and behavior. Reading, MA: Addison-Welsey Publishing Co. 1975.

[37] Ajzen I. From intentions to actions: A theory of planned behavior. In: Action control. Springer; 1985:11-39.

[38] Meg Gerrard et al., A Dual-Process Approach to Health Risk Decision Making: The Prototype Willingness Model, 28(1) Developmental Review29 (2008).

[39] McKelvey, K., Popova, L., Pepper, J., Brewer, N., Halpern-Felsher, B.  Adolescents Have Unfavorable Opinions of Adolescents Who Use E-cigarettes.  In Review.

[40] Gerrard, M., Gibbons, F.X., Benthin, A.C., & Hessling, R.M. (1996). A longitudinal study of the reciprocal nature of risk behaviors and cognitions in adolescents: What you do shapes what you think and vice-versa. Health Psychology, 15, 344-354; Meg Gerrard et al., A Dual-Process Approach to Health Risk Decision Making: The Prototype Willingness Model, 28(1) Developmental Review29 (2008).

[41] Gerrard M, Gibbons FX, Stock ML, Lune LS, Cleveland MJ. Images of smokers and willingness to smoke among African American pre-adolescents: An application of the prototype/willingness model of adolescent health risk behavior to smoking initiation. Journal of Pediatric Psychology. 2005 Feb 23;30(4):305-18.

[42] Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T

[43] Morrell, H. E. R. (20101011). Predicting adolescent perceptions of the risks and benefits of cigarette smoking: A longitudinal investigation. Health Psychology. https://doi.org/10.1037/a0021237

[44] Song, A. V., Morrell, H. E. R., Cornell, J. L., Ramos, M. E., Biehl, M., Kropp, R. Y., & Halpern-Felsher, B. L. (2009). Perceptions of Smoking-Related Risks and Benefits as Predictors of Adolescent Smoking Initiation. American Journal of Public Health, 99(3), 487–492. https://doi.org/10.2105/AJPH.2008.137679.

[45] Halpern-Felsher, BL, Biehl, M, Kropp, RY, & Rubinstein, ML. Perceived risks and benefits of smoking: Differences between adolescents with different smoking experiences and intentions. Preventive Medicine. 2004 Sep; 39(3): 559-567. PMID: 15313096

[46] Kim, M., Watkins, S. L., Koester, K. A., Mock, J., Kim, H. C., Olson, S., … & Ling, P. M. (2020). Unboxed: US Young Adult Tobacco Users’ Responses to a New Heated Tobacco Product. International Journal of Environmental Research and Public Health17(21), 8108.

[47] Schane RE, Glantz SA, Ling PM. Nondaily and social smoking: an increasingly prevalent pattern. Arch Intern Med. 2009 Oct 26;169(19):1742-4. doi: 10.1001/archinternmed.2009.315. PMID: 19858429; PMCID: PMC4350771.

[48] Schane RE, Glantz SA, Ling PM. Social smoking implications for public health, clinical practice, and intervention research. Am J Prev Med. 2009 Aug;37(2):124-31. doi: 10.1016/j.amepre.2009.03.020. PMID: 19589449; PMCID: PMC2771192.

[49] Schane RE, Glantz SA. Education on the dangers of passive smoking: a cessation strategy past due. Circulation. 2008 Oct 7;118(15):1521-3. doi: 10.1161/CIRCULATIONAHA.108.805259. PMID: 18838572; PMCID: PMC2760983.

[50] Schane RE, Prochaska JJ, Glantz SA. Counseling nondaily smokers about secondhand smoke as a cessation message: a pilot randomized trial. Nicotine Tob Res. 2013 Feb;15(2):334-42. doi: 10.1093/ntr/nts126. Epub 2012 May 16. PMID: 22592447; PMCID: PMC3545714.

[51] Song, AV, Glantz, S, Halpern-Felsher, BL.  Perceptions of Second-hand Smoke Risks Predict Future Adolescent Smoking Initiation. Journal of Adolescent Health. 2009 Dec; 45 (6):618-625. PMID: 19931835.

[52] Wang, T. W., Gentzke, A. S., Creamer, M. R., & et al. (2019). Tobacco Product Use and Associated Factors Among Middle and High School Students—United States, 2019. MMWR Surveillance Summaries, 68(12). Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6903396/

[53] Cornelius ME, Wang TW, Jamal A, L. C., & Neff LJ. Tobacco Product Use Among Adults — United States, 2019. MMWR Morb Mortal Wkly Rep 2020;69:1736–1742. DOI: http://dx.doi.org/10.15585/mmwr.mm6946a4external.

[54] Xie W, Kathuria H, Galiatsatos P, et al. (2020). Association of Electronic Cigarette Use with Incident Respiratory Conditions Among US Adults From 2013 to 2018. JAMA Network Open, 3(11), e2020816-e2020816. doi:10.1001/jamanetworkopen.2020.20816

[55] Wills, T. A., Soneji, S. S., Choi, K., Jaspers, I., & Tam, E. K. (2020). E-cigarette Use and Respiratory Disorder: An Integrative Review of Converging Evidence from Epidemiological and Laboratory Studies. European Respiratory Journal, 1901815. doi:10.1183/13993003.01815-2019

[56] CDC. Current Cigarette Smoking Among Adults in the United States. Centers for Disease Control and Prevention. Published November 16, 2020. Accessed November 28, 2020. https://www.cdc.gov/tobacco/data_statistics/fact_sheets/adult_data/cig_smoking/index.htm

[57] Cornelius ME, Wang TW, Jamal A, L. C., & Neff LJ. Tobacco Product Use Among Adults — United States, 2019. MMWR Morb Mortal Wkly Rep 2020;69:1736–1742. DOI: http://dx.doi.org/10.15585/mmwr.mm6946a4external.

[58] Watkins, S.L., Thrul., J., Max, W., & P.M. Ling (2019). Real-world effectiveness of smoking cessation strategies for young and older adults: Findings from a nationally representative cohort. Nicotine & Tobacco Research. 22(9): 1560-1568. 10.1093/ntr/ntz223   

[59] U.S. Department of Health and Human Services. Smoking Cessation. A Report of

the Surgeon General. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, 2020. Available: https://www.cdc.gov/tobacco/data_statistics/sgr/2020-smoking-cessation/index.html  

[60] Lempert LK, Glantz S. Analysis of FDA’s IQOS marketing authorisation and its policy impacts. Tob Control. 2020 Jun 29:tobaccocontrol-2019-055585. doi: 10.1136/tobaccocontrol-2019-055585. Epub ahead of print. PMID: 32601147.

[61] FDA. PMTA Coversheet: Technical Project Lead Review (TPL), April 29, 2019. Available: https://www.fda.gov/media/124247/download

[62] Lempert LK, Glantz S. Analysis of FDA’s IQOS marketing authorisation and its policy impacts. Tob Control. 2020 Jun 29:tobaccocontrol-2019-055585. doi: 10.1136/tobaccocontrol-2019-055585. Epub ahead of print. PMID: 32601147.

[63] Flick, U. (Ed.). (2013). The SAGE handbook of qualitative data analysis. Sage. Available: https://methods.sagepub.com/book/the-sage-handbook-of-qualitative-data-analysis

[64] CASP Qualitative Studies Checklist. Available: https://casp-uk.net/casp-tools-checklists/

[65] Morse, J. M. (2003). A review committee’s guide for evaluating qualitative proposals. Qualitative health research13(6), 833-851. https://doi.org/10.1177/1049732303013006005

[66] Allison Tong, Peter Sainsbury, Jonathan Craig, Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups, International Journal for Quality in Health Care, Volume 19, Issue 6, December 2007, Pages 349–357, https://doi.org/10.1093/intqhc/mzm042

[67] Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative inquiry16(10), 837-851. https://doi.org/10.1177/1077800410383121

[68] Saldaña, Johnny. The Coding Manual for Qualitative Research. Sage 2009

https://us.sagepub.com/en-us/nam/the-coding-manual-for-qualitative-researchers/book243616

[69] Glantz, SA. PMI’s Own Data on Biomarkers of Potential Harm in Americans Show that IQOS is Not Detectably Different from Conventional Cigarettes, so FDA Must Deny PMI’s Modified Risk Claims, Docket No.  FDA-2017-D-3001, November 13, 2017. Available: https://tobacco.ucsf.edu/pmi%E2%80%99s-own-data-biomarkers-potential-harm-americans-show-iqos-not-detectably-different-conventional-cigs

[70] Glantz, SA. Letter to the FDA Center for Tobacco Products, December 21, 2018. PMI’s 6-month study, “Evaluation of Biological and Functional Changes in Healthy Smokers After Switching to THS 2.2 for 26 Weeks (ZRHR-ERS-09 US) submitted in PMI IQOS MRTP June 8, 2018 amendment to FDA-2017-D-3001-0002” does not support claims of reduced risk. Available: https://tobacco.ucsf.edu/sites/g/files/tkssra4661/f/wysiwyg/Public%20comment%20on%20PMI%206%20month%20study.pdf

[71] FDA, Applications for Premarket Review of New Tobacco Products: Guidance for Industry, Draft Guidance (September 2011). Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/applications-premarket-review-new-tobacco-products

[72] FDA, Premarket Tobacco Product Applications for Electronic Nicotine Delivery Systems, Guidance for Industry (June 2019). Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/premarket-tobacco-product-applications-electronic-nicotine-delivery-systems-ends

[73] McKelvey, K. & Halpern-Felsher, BL. Adolescent cigarette-smoking perceptions and behavior: Tobacco control gains and gaps amidst the rapidly expanding tobacco market 2001-2015. Journal of Adolescent Health. 2017 Feb;60(2):226-228. PMID: 27939880

[74] McQuoid, J., Keamy-Minor, E., & Ling, P. M. (2020). A practice theory approach to understanding poly-tobacco use in the United States. Critical Public Health30(2), 204-219.

[75] Lempert LK, Glantz S. Analysis of FDA’s IQOS marketing authorisation and its policy impacts. Tob Control. 2020 Jun 29:tobaccocontrol-2019-055585. doi: 10.1136/tobaccocontrol-2019-055585. Epub ahead of print. PMID: 32601147.

[76] FDA. Premarket Tobacco Product Applications for Electronic Nicotine Delivery Systems, Guidance for Industry (June 2019). Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/premarket-tobacco-product-applications-electronic-nicotine-delivery-systems-ends

[77] FDA. Modified Risk Tobacco Product Applications, Draft Guidance (March 2012). Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/modified-risk-tobacco-product-applications

[78] Institute of Medicine 2012. Scientific Standards for Studies on Modified Risk Tobacco Products. Washington, DC: The National Academies Press. https://doi.org/10.17226/13294. Available: https://www.nap.edu/download/13294

[79] FDA. Marketing Order, FDA Submission Tracking Numbers (STNs): PM0000424-PM0000426, PM0000479, April 30, 2019. Available: https://www.fda.gov/media/124248/download&nbsp;

[80] FDA. Marketing Order, FDA Submission Tracking Numbers (STNs): PM0000491 and PM0000492, December 17, 2019. Available: https://www.fda.gov/media/133635/download

[81] FDA. Modified Risk Orders – Exposure Modification, FDA Submission Tracking Numbers: MR0000059‐MR000061, MR0000133, July 7, 2020. Available: https://www.fda.gov/media/139797/download

Published by Stanton Glantz

Stanton Glantz is a retired Professor of Medicine who served on the University of California San Francisco faculty for 45 years. He conducts research on tobacco and cannabis control and cardiovascular disease/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: