Mitigating the Youth Vaping Epidemic through Increasing Screening Rates for Youth Vaping/E-Cigarette Use in Pediatrics



Wynnyee Tom MD1; Brooke Harris PhD2; Arthur Shen MPH3; Lizeth Rivas BS4; Denise Williams MA5; Seham El-diwany MD1; Edan Wernik MD6; Andrew Nguyen1; Gabriella Jones1; Chynna Bantug MD1


Perm J 2020;25:20.064 [Full Citation]
E-pub: 12/30/2020


Introduction: E-cigarette/vaping use in adolescents has increased 77.8% among high schoolers and 48.5% among middle schoolers in 2017-2018. As such, there is need for an effective workflow for screening for vaping. We aimed to increase screening rates of e-cigarette/vaping users from less than 1% to at least 50% in 6 months.

Methods: Screening for vaping in youth was implemented in a pediatric clinic in Northern California beginning in the summer of 2019 for 6 months. Depending on comorbidity, severity, and readiness to quit, patients were referred to treatment. Outcomes included screening rates, process measure included positive screening rates, and balancing measure was provider time.

Results: The clinic completed 1414 physicals with an average screening rate of 76% and a positive rate of 7.9%. The average age of patients was 15 (standard deviation = 1.3), 48% were female and 29% were Asian/Pacific Islander, 23% Hispanic, and 23% White. After 6 months, we met our goal in all but 1 plan-do-study-act (PDSA) cycle.

Discussion: We created a standardized workflow that identified teens who vaped. When compared to other studies, the positive rate for this study was low which is likely due to misinterpretation by staff of screening questions as well as the fact that data were collected during a clinic visit.

Conclusions: It is important to find ways in which providers can mitigate this epidemic given the alarming increase in e-cigarette/vaping use among adolescents. This study furthers the effort to develop a screening method that is simple and brief, allowing physicians to intervene if necessary.


National surveys have identified an alarming trend in the use of e-cigarette and vaping devices. From 2017 to 2018, e-cigarette use increased 77.8% among high schoolers and 48.5% among middle schoolers over 1 year.1 Data from 2017 indicated that 42.2% of 9th-12th graders tried using an e-cigarette/vaping device, which are more commonly referred to as “vape pens,” “electronic cigarettes,” brand “JUUL,”2 and new generations of devices such as “mods” and “Puff Bars.”

In Santa Clara County, the California Student Tobacco Survey (2017-2018) found that 13.2% of Santa Clara County teens reported using e-cigarettes in the past month, classifying them as current users. Nearly 1 in 3 Santa Clara county teens report that they have used e-cigarettes at least once.3

As the popularity of these devices increase, there is an increased risk of nicotine addiction, toxic effects from nicotine overuse, cancer-causing chemicals, secondhand exposure, progression to smoking cigarettes (with studies reporting a 30% chance of smoking traditional cigarettes within 6 months after using an e-cigarette/vaping device for the first time).4 In recent studies by the CDC on the current outbreaks, risks now also encompass such acute occurrences such as lung injury (EVALI), vaping-associated pumonary injury (VAPI) and death.5

To our knowledge, there is very little published on implementation of screening for teen vaping. When analyzing local hospital systems and Santa Clara Public Health, there was not a uniform workflow in place as of December 2019. Similarly, there was a lack of information regarding types of questions to ask in such workflows.

To this end, the objective of this quality improvement project was to develop a uniform screening workflow to be used in the identification of e-cigarette/vaping users among 13- to 17-year-olds. We aimed to increase screening rates for e-cigarette/vaping users from < 1% to ≥ 50% over 6 months.


We used quality improvement methodology to rollout screening in a community-based pediatric clinic with 30 physicians and 1 nurse practitioner in Northern California for 6 months beginning in the summer of 2019. The pediatrics department is separated into 3 modules. Screening was rolled out first in module C for 9 weeks to fine tune the workflow, then rolled out in module B at week 10, and finally, implemented in module A at week 15. Prior to implementation, this project was deemed “not human subjects research” by the institution’s Not Human Subjects Determination Committee because the activity does not meet the regulatory definition of research as defined by the code of federal regulations [45 CFR 46.102(d)].

Because this workflow was new, the primary outcome was to measure screening rates. We obtained a 3-month baseline screening rate prior to rollout via retrospective chart review and then collected weekly screening rates via data extraction from the Electronic Health Record and through review of the paper questionnaires. The team also tracked the number of positive screens as a process measure and compared them to existing national and county prevalence rates to confirm the utility of the question. Qualitative feedback from medical assistants (MAs) and physicians along with clinic leadership about time and impact to care were obtained as a balancing measure. Finally, we collected basic demographics to characterize our patient population.

We selected a screening rate goal that was suggested by our key stakeholders (eg, department staff and leaders) as reasonable to achieve. Prior experience with other new screening measures implemented in the clinic suggested that a rate of 50% was a reasonable threshold. As such, our goal was set at an average screening rate of ≥ 50% over 6 months. To improve screening rates, we conducted 10 “plan-do-study-act” (PDSA) cycles over 6 months across the 3 modules. Interventions were mostly related to adjustments to the questions (see Table 1 for more information on interventions delivered via PDSAs). The process for selecting interventions included weekly meetings with the project team to discuss key stakeholder and clinical staff feedback, to review the primary outcome, to track progress, and to conduct group discussions over possible solutions to improve screening rates.

Table 1. Intervention timeline by plan, do, study, act (PDSA) cycle

Week Screening Rate (%) Positive Rate (%) Intervention Screening Questions
1 73  21 PDSA 1: Module C went live and started screening visits with cohort 1 question. Have you ever used tobacco (smoke, chew, e-cigarettes) or other vapor product? yes/no
2 60 10 PDSA 2: Reminder stickers added to each computer monitor. Change: stopped collecting paper surveys
3 43 5 PDSA 3: Module C changed to cohort 2 question. Have you ever used vapor products or e-cigarettes? yes/no
4-5 55 7 PDSA 4: Reinstate the collection of the paper survey
6-9 68 12 PDSA 5: MA training refresher
10-13 75 7 PDSA 6: Module B went live with cohort 2 question.
14 81 6 PDSA 7: Module B changed to cohort 3 questions. Have you ever used vapor products or e-cigarettes or JUULed? yes/no
15 83 7 PDSA 8: Module A went live with cohort 3 questions. When was the last time you vaped?
16-21 79 10 PDSA 9: Module C changed to cohort 3 questions. Type?
How often do you vape?
22-26 84 6 PDSA 10: All modules changed to cohort 4 questions. The next questions will ask you about vaping. By vaping we mean JUULing, using vape pens, or e-cigarettes.
How often do you vape? Daily/weekly/monthly/rarely/never
When was the last time you vaped?
What did you use?

Bolded answer choice indicates positive screen. MA = Medical Assistant


When the patient arrived for an appointment, the receptionist gave the patient a paper questionnaire to complete. All patients between the ages 13 to 17 years old scheduled for a physical who were English speakers and who were physically, mentally, and emotionally able to complete the screener (as determined by the parent) were offered the opportunity to complete the screener in a confidential setting. We targeted 13- to 17-year-old teens because we decided to embed our screener within the clinic's current teen questionnaire to use an existing workflow. It is clinic policy to only assess this age group using this paper questionnaire because of state law related to confidentiality and minor consent.

The locally developed teen questionnaire assessed wellness concerns such as mental health, nutrition, activity, body image, safety, and sexual health in addition to substance use, and we embedded the vaping screener in the “substance use” section of this teen questionnaire (See Table 1 and “Screening Questions” section below for details). The vaping screener was in addition to the traditional screening questions for substance abuse (that includes smoking/chewing tobacco and smoking marijuana) to allow us to better assess vaping as a mode of transmission for these substances.6

Table 2. Demographics by Cohort of Teens who Received Different Versions of the Screening Question

  Cohorts Total
  1 2 3 4  
Study week 1-2 3-13 14-21 22-26  
Department module C C and B All All All
Population (teens coming in for visit) 107 494 473 340 1,414
Screens (teens screened for vaping) 66 348 379 280 1,073
Positives (among those screened) 9 23 36 17 85
% Screening rate (among the teen visit) 61.7 70.4 80.1 82.4 75.9
% Positive rate (among those screened) 13.6 6.6 9.5 6.1 7.9
Average age (of teens coming in for visit) 15.02 14.95 14.99 15.16 15.02
Age standard deviation 1.32 1.36 1.38 1.27 1.34
% Female (of teens coming in for visit) 55 44 48 50 49
% Race breakdown (of teens coming in for visit)        
 Asian/Pacific Islander 35 27 27 32 29
 Hispanic/Latino 15 23 28 19 23
 White 29 20 20 26 23
 Other 17 19 18 14 17
 Blank/unknown/decline 5 5 5 5 5
 Black -- 4 3 3 3
 American Indian/Alaska Native -- 1 < 1 < 1 < 1

The MA reviewed the results of the questionnaire with the adolescent alone in the clinic room, gave the teen a pocket card that included information about e-cigarette/vaping use (whether they vaped or not) and resources for quitting (eg, numbers for local specialty services,, Truth Initiative’s Text QUIT to 706-222-QUIT, California Smokers’ Healthline, and newer California Vaper’s hotline), and the MA informed the provider if the vaping question was endorsed. The additional vaping question from the paper questionnaire was then entered with a prepopulated phrase by the medical assistant into the electronic medical records (EMR) and the paper questionnaire was collected (see Figure 1 for the workflow).


Figure 1. Youth Vaping Screening Workflow.

If positive, providers inquired about the severity of vaping by conducting the CRAFFT screener 7 and assessed the teen’s readiness to quit using the readiness ruler.8 Depending on comorbidities (eg, depression, anxiety, eating disorder, etc), severity (eg, weekend or social user versus daily user), and readiness to quit (ie, 7-10 on the readiness ruler as set by the Teen Clinic Provider), the teen was either referred to child psychiatry, addiction treatment, or the teen clinic for a vaping cessation intervention.

Screening Questions

The first PDSA cycle included the yes/no question: “Have you ever used tobacco (smoke, chew, e-cigarettes) or other vapor product?” Subsequent PDSA cycles included the following questions: PDSA #3: “Have you ever used vapor products or e-cigarettes?”; PDSA 7: “Have you ever used vapor products or e-cigarettes or JUULed? When was the last time you vaped? Type? How often do you vape?”; PDSA 10: “The next questions will ask you about vaping. By vaping we mean JUULing, using vape pens, or e-cigarettes. How often do you vape? When was the last time you vaped? What did you use?” A positive screen was identified if the teen said “yes” to ever vaping or indicated that they vape daily, weekly, monthly, or rarely for the “How often do you vape?” question. Changes to questions were made in response to key stakeholder feedback, to updates from the literature, to better determine type of intervention needed, and to improve identification of positive screens. Other interventions included adding reminder stickers to clinic workstations, training refreshers for clinical staff, rolling out screening to different modules, and stopping and re-starting collection of paper data (see Table 1 for more information).


The clinic completed 1414 physical exams over 6 months. Patients who were screened were on average 15 years old (mean [SD] age = 15.02 [1.3] years), relatively evenly split for gender (48% female), and relatively diverse (Asian/Pacific Islander = 29%; Hispanic = 23%; and White = 23%) (See Table 2 for a complete breakdown). After 6 months, it was determined that we met our goal (ie, a screening rate of ≥ 50%) in all but 1 PDSA cycle. See Figure 2 for a graphical representation of our screening and positive rates across PDSA cycles. See Table 1 for a list of PDSA interventions. Our first PDSA cycle included a screening rate of 73% using the question “Have you ever used tobacco (smoke, chew, e-cigarettes) or other vapor product?”; subsequent screening rates varied between a low of 43% in cycle 3 using the question “Have you ever used vapor products or e-cigarettes?” to a high of 84% in cycle 10 using the question “The next questions will ask you about vaping. By vaping we mean JUULing, using vape pens, or e-cigarettes. How often do you vape?” for an overall average screening rate of 74%.

Of those who were screened (n = 1073), 85 teens scored positive for an overall positive rate of 7.9%, with the highest positive rate (21%) occurring in cycle 1 and the lowest positive rate occurring in cycle 3 (5%) (See Table 2 for average screening and positive rates, and Figure 2 for screening and postive rates per PDSA cycle). Related to our balancing measure (ie, informal, qualitative feedback from clinic staff and providers about impact of screening to their time with patients), clinical staff reported screening only added 30 seconds to 1 minute of their time with the patient. Because they felt that it was an important topic, they were happy to incorporate review of the screener to their workflow.


Figure 2. Screening and positive rates by PDSA cycle.


This study yielded several key conclusions regarding the ramifications of using an e-cigarette/vaping screening procedure in pediatrics. We successfully created a standardized workflow that identified teens that use e-cigarette/vaping with an overall positive rate of 7.9%. Facilitators to screening included having a standardized workflow, key stakeholder support, and sense of urgency regarding this epidemic. The severity of the aforementioned EVALI outbreak also helped with the cause.

When compared to other e-cigarette/vaping studies, the positive rate for this study was low for lifetime users of e-cigarette/vaping but within range for current users with a reported lifetime prevalence of 31% and 13% for current use.9,10 This is may be due to misinterpretation of the question by staff or the fact that the question was ask in a way where the teen would receive follow-up if the question was endorsed. Specifically, staff indicated that when a teen endorsed vaping once (that is a positive), they recorded the answer to the screen as a “no.” Data supported this conclusion, because our highest positive rate was in week 1 (21%) when the clinic staff was the least experienced with the workflow. Most surveys reporting higher rates were conducted using anonymous methods, and to our knowledge, no other studies have used similar methods for screening teens.


Although the screening procedure outlined is ongoing, the study spanned 6 months. Hence, long-term data collection is necessary for a more extensive understanding of the implications of using such a screening procedure. As with any study involving patient responses, the issue of response integrity is another factor which can skew results. Regardless of measures taken, teens may have altered responses for concern that intervention would involve contacting their legal guardians. Another study investigating the effectiveness of e-cigarette/vaping screening explains similar concerns, underlining limitations such as patient response bias.11 Finally, screening only occurred during annual physicals; possibly missing teens that come to the clinic for urgent visits. This is especially concerning for teens complaining of respiratory-related conditions.

Future Studies

E-cigarette/vaping screening workflows in pediatrics are relatively new as providers attempt to develop cohesive and effective cessation pathways. The findings from this study provide a basis for other studies which may be explored in the future. Future directions include replication of the study in other clinics or medical centers to support external validity and further assess positive rates, refinement of screening questions, and development of a manualized treatment to support vaping cessation for teens. In future studies, additional questions about current use and type of substances used will be considered. This will help the study team further standardize the workflow, so all teens receive similar interventions. Although this study focused specifically on the screening, a similar study can be conducted to find optimal methods of treatment for patients who screen positive.

Disclosure Statement

The authors have no conflicts of interest to disclose.


CRAFFT = car, relax, alone, forget, friends, trouble; ENDS = electronic nicotine delivery system; EVALI = E-cigarette or vaping product use associated lung injury; MA = medical assistant; PDSA = plan-do-study-act; VAPI = vaping-associated pulmonary injury


Special thanks to San Jose Kaiser Permanente Pediatric Team for all the support and being the pilot site and Sian Aynsley, our Library Manager, for help searching the extant literature. The corresponding author affirms that she has listed everyone who contributed significantly to the work on this manuscript. Finally, we wish to acknowledge that this work has been presented as a poster at the Institute for Health care Improvement National Forum on Quality Improvement in Health Care in December 2019 in Orlando, FL.

Author Affiliations

1Department of Pediatrics, Kaiser Permanente San Jose Medical Center, San Jose, CA

2Department of Graduate Medical Education, Kaiser Permanente San Jose Medical Center, San Jose, CA

3Department of Strategic Initiatives and Technology, Kaiser Permanente San Jose Medical Center, San Jose, CA

4Department of Member Outreach, Kaiser Permanente San Jose Medical Center, San Jose, CA

5Department of Medical Office Controllers, Kaiser Permanente San Jose Medical Center, San Jose, CA

6Department of Family Medicine, Kaiser Permanente San Jose Medical Center, San Jose, CA

Corresponding Author

Wynnyee Tom, MD (

How to Cite this Article

Tom W, Harris B, Shen A, et al. Mitigating the youth vaping epidemic through increasing screening rates for youth vaping/e-cigarette use in pediatrics. Perm J 2020;25:20.064. DOI: 10.7812/TPP/20.064


1. Gentzke AS, Creamer M, Cullen KA, et al. Vital signs: tobacco product use among middle and high school students — United States, 2011–2018. MMWR Morb Mortal Wkly Rep 2019 Feb;68:157–64. DOI:

2. Kann L, McManus T, Harris WA, et al. Youth risk behavior surveillance — United States, 2017. MMWR Surveill Summ 2018 Jun;67(8):1-114. DOI:

3. Zhu S-H, Lee J, Zhuang Y-L, Braden K, Cole AG, Gamst A. Tobacco use among high school students in Santa Clara county: Findings from the 2017-18 California student tobacco survey - 2019.

4. Leventhal AM, Strong DR, Kirkpatrick MG, et al. Association of electronic cigarette use with initiation of combustible tobacco product smoking in early adolescence. J Am Med Assoc 2015 Aug;314(7):700-7. DOI:

5. Butt YM, Smith ML, Tazelaar HD, et al. Pathology of vaping-associated lung injury. N Engl J Med 2019 Oct;381:170-1. DOI:

6. Kelly SM, Gryczynski J, Mitchell SG, Kirk A, O’Grady KE, Schwartz RP. Validity of brief screening instrument for adolescent tobacco, alcohol, and drug use.Pediatrics May 2014 May;133:819-26. DOI:,

7. Mitchell SG, Kelly SM, Gryczynski J, et al. The CRAFFT cut-points and DSM-5 criteria for alcohol and other drugs: A reevaluation and reexamination. Subst Abus 2014 July;35(4):376–80. DOI:

8. Gold MA, Kokotailo PK. Motivational interviewing strategies to facilitate adolescent behavior change. Adolescent Health Update 2007;20(1):1-10.

9. Cullen KA, Ambrose BK, Gentzke AS et al. Notes from the field: Use of electronic cigarettes and any tobacco product among middle and high school students-United States, 2011–2018. MMWR (Morb Mortal Wkly Rep) 2018 Nov;6;67(45):1276. DOI:

10. Santa Clara County Public Health. Almost 1 in 3 Santa Clara county teens have tried vaping. Accessed October 10, 2019. Available at:

11. Perikleous, EP, Steiropoulos, P, Paraskakis, E, et al. E-cigarette use among adolescents: an overview of the literature and future perspectives. Frontiers in public health 2018 Mar;6:86. DOI:

Keywords: adolescent, e-cigarette, mods, PDSA, Puff Bars, screening, teens, vaping, vaporizing, workflow


Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.


2 million page views of TPJ articles in PubMed from a broad international readership.


Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, and CrossRef.




ISSN 1552-5775 Copyright © 2021

All Rights Reserved