Validation Study of Kaiser Permanente Bedside Dysphagia Screening Tool in Acute Stroke Patients



 

Barbara Schumacher Finnegan MA, RN, CPHQ1; Melissa M Meighan DNP, MS, RN, CNRN, SCRN, NEA-bc2; Noelani C Warren MSN, RN, SCRN3; Meghan K Hatfield MPH4; Stacey Alexeeff PhD4; Jorge Lipiz MD5; Mai Nguyen-Huynh MD, MAS46

Perm J 2020;24:19.230 [Full Citation]

https://doi.org/10.7812/TPP/19.230
E-pub: 12/02/2020

ABSTRACT

Introduction: Dysphagia occurs in up to 50% of patients with acute stroke symptoms, resulting in increased aspiration pneumonia rates and mortality. The purpose of this study was to validate a health system’s dysphagia (swallow) screening tool used since 2007 on all patients with suspected stroke symptoms. Annual rates of aspiration pneumonia for ischemic stroke patients have ranged from 2% to 3% since 2007.

Methods: From August 17, 2015 through September 30, 2015, a bedside dysphagia screening was prospectively performed by 2 nurses who were blinded to all patients age 18 years or older admitted through the emergency department with suspected stroke symptoms at 21 Joint Commission accredited primary stroke centers in an integrated health system. The tool consists of 3 parts: pertinent history, focused physical examination, and progressive testing from ice chips to 90 mL of water. A speech language pathologist blinded to the nurse’s screening results performed a formal swallow evaluation on the same patient.

Results: The end study population was 379 patients. Interrater reliability between 2 nurses of the dysphagia screening was excellent at 93.7% agreement (Ƙ = 0.83). When the dysphagia screenings were compared with the gold standard speech language pathologist professional swallow evaluation, the tool demonstrated both high sensitivity (86.4%; 95% confidence interval = 73.3-93.6) and high negative predictive value (93.8%; 95% confidence interval = 87.2-97.1).

Conclusion: : This tool is highly reliable and valid. The dysphagia screening tool requires minimal training and is easily administered in a timely manner.

INTRODUCTION

This study was performed to test the validity, sensitivity, and positive predictive value of a bedside screening tool to detect risk of aspiration pneumonia due to dysphagia. Difficulty swallowing or dysphagia is common among acute stroke patients, occurring in approximately 30% to 78% of the population.1-3 About 50% of patients aspirate, and one-third of those who aspirate develop pneumonia.3-5 Dysphagia is associated with a 3-fold increase in mortality risk, mainly attributable to pneumonia.3,4 Other adverse consequences of dysphagia may include dehydration, malnutrition, weight loss, increased morbidity, increased length of stay, reduced rehabilitation capabilities, and psychological illness.2,6 Rapid screening for dysphagia by a bedside nurse was developed to optimize resource utilization and to improve patient satisfaction. Speech language pathology services are limited. Waiting for these services may unnecessarily lengthen the time the patient is without oral intake.7

Dysphagia screening tools have been developed, such as the Barnes Jewish Hospital Stroke Dysphagia Screen, the Modified Mann Assessment of Swallowing Ability, the Emergency Physician Swallowing Screening, the Toronto Bedside Swallowing Screening Test,8 and Gugging Swallow Screen.9 Agreed-upon criteria for a dysphagia screening tool include adequate sensitivity and specificity to identify dysphagia and aspiration risk, a scoring system of “pass/fail,” ease of use for health care providers other than a speech language pathologist (SLP), and the ability to rescreen with a change in clinical condition.10,11 The use of dysphagia protocols to reduce the risk of pneumonia, death, or dependency has not been proven; however, the American Heart Association 2018 Guidelines for Early Management of Acute Stroke Guidelines state that it is reasonable to screen patients before oral intake.12,13

The bedside dysphagia screening tool used at the large integrated health system was developed in 2007. The annual aspiration rates for ischemic stroke patients have averaged 2% to 3% over a period of 12 years. The tool incorporates components of the Massey Bedside Swallowing Screen and the Burke dysphagia screening tool.14,15 The dysphagia screening tool is divided into 3 parts. Part I includes selected history with a failed test for a tracheostomy, history of aspiration pneumonia, or currently nothing by mouth (NPO) with tube feeding. Part II includes a cognitive and motor skills assessment by the registered nurse (RN). Patients must effectively swallow their own secretions, move the tongue, smile and pucker the lips, and cough on command. In Part III, the patient must swallow without coughing or throat clearing during and 1 minute after taking ice chips. Then the patient may progress to drinking 90 mL of water continuously (Figure 1). The patient may fail the screening at any point or may pass and be allowed oral intake.

tpj19230f1

Figure 1. Dysphagia screening tool.

Inclusion/Exclusion Criteria

Study inclusion criteria were all patients 18 years or older exhibiting suspected stroke symptoms who were admitted to the emergency department (ED) at 1 of the 21 participant hospitals and had a diagnosis of stroke or transient ischemic attack (TIA) upon discharge. Exclusion criteria were a history of dysphagia, pre-existing gastrostomy tubes, being currently intubated, having had received alteplase upon admission, and not having a diagnosis other than stroke or TIA upon discharge. Patients who received alteplase were excluded because they may change rapidly between the RN screening and evaluation by speech therapy.

METHODS

This study received approval from the governing Institutional Review Board. Informed consent was waived because both the dysphagia screening tool and speech language therapy evaluation were considered standard of care. The study was conducted from August 17, 2015 through September 30, 2015. The dysphagia screening tool was prospectively performed by 2 ED nurses who were blinded. The first nurse recorded the results of the dysphagia screening on a paper form. A second nurse completed a second dysphagia screening within 1 hour and recorded the results in the electronic medical record. The goal was to conduct both screens within a short period of time. An SLP consult was requested for all patients screened. The SLPs were asked to complete their dysphagia evaluation within 12 hours. Per usual practice, the patient was allowed oral intake before the SLP evaluation if both screenings were passed. If either dysphagia screening was failed, the patient was kept NPO until the SLP evaluation was completed. The SLP was blinded to the results of the screening and did not review the electronic medical record. In usual practice, they do not use the presence of food or drink at the bedside as an indication of NPO status because family members may unknowingly bring in food or give their loved ones water.

Results of the first and second dysphagia screenings were entered into a database. The portion of the screening prompting a fail result was recorded. The results of the SLP formal swallow evaluation, including the type of liquids tolerated by the patient (eg, honey thick, nectar thick, or nectar thin), were entered. For purposes of this study, only tolerance of thin liquids was considered a passing score; tolerance of all other modified liquids was considered to be a fail. Validity was determined by a comparison of the second RN dysphagia screening to the results of the SLP dysphagia evaluation.

Statistics were calculated using a SAS computer program. Reproducibility of the dysphagia screening was quantified by the percent agreement and the interrater reliability. To assess the performance of the dysphagia screening in comparison to the formal assessment by an SLP, both the sensitivity and negative predictive value were quantified. The specificity and positive predictive value (PPV) were analyzed. The point estimate and 95% confidence interval (CI) were determined.

RESULTS

There were 726 patients screened using the Dysphagia Screening Tool. After applying the exclusion criteria, 392 stroke patients were enrolled. The average age was 70 ± 14 years, and 48% were female (n = 188). The group was composed of 52% (n = 203) white non-Hispanic, 12% (n = 45) Black, 11% (n = 44) Asian/Pacific Islander, 20% (n = 78) Hispanic, and 6% (n = 22) Other/Unknown (Table 1). Members of the health system constituted 86% (n = 338) of the population. Seventy-one percent (n = 278) of subjects had experienced an ischemic stroke, 25% had experienced a TIA (n = 96), and 5% had experienced a hemorrhagic stroke (n = 18). The screening was done on all patients presenting with suspected stroke symptoms. The median National Institutes of Health Stroke Scale score was 1, with a range of 0 to 23. Sixty-nine percent (n = 272) of patients were discharged home, and 9% (n = 34) had home health assistance. Four percent (n = 15) were discharged to a rehabilitation facility and 7% (n = 28) to a skilled nursing home. Less than 1% (n = 3) of patients died (Table 1).

Table 1. Demographic information

Characteristics Received swallow screening and evaluation (n = 169) Received swallow screenings (n = 223) p value  Overall (n = 392)
Total (%) Total (%)   Total (%)
Patient characteristics
 Female, % 74 (43.8) 114 (51.1) 0.15 188 (48.0)
         
 Age (mean ± SD) 72 ± 13 68 ± 14 0.01 70 ± 14
Race, %     0.41  
  White, non-Hispanic 85 (50.3) 118 (52.9)   203 (51.8)
  Black 18 (10.6) 27 (12.1)   45 (11.5)
  Asian/Pacific Islander 16 (9.5) 28 (12.6)   44 (11.2)
  Hispanic 41 (24.3) 37 (16.6)   78 (19.9)
  Other/unknown 9 (5.3) 13 (5.8)   22 (5.6)
 NIHSS     <0.001  
  Mean ± SD 4 ± 5 2 ± 3   3 ± 4
  Median 3 1   1
  Range 0-23 0-23   0-23
Case characteristics
 Discharge status     <0.001  
  Home 83 (49.1) 155 (69.5)   238 (60.7)
  Home health 22 (13.0) 12 (5.4)   34 (8.7)
  Rehab facility 11 (6.5) 4 (1.8)   15 (3.8)
  Skilled nursing facility 20 (11.8) 8 (3.6)   28 (7.1)
  Expired 2 (1.2) 1 (0.4)   3 (0.8)
  Other 31 (18.3) 43 (19.3)   74 (18.9)
 Type of stroke     0.07  
  Ischemic 130 (76.9) 148 (66.4)   278 (70.9)
  Hemorrhagic 6 (3.6) 12 (5.4)   18 (4.6)
  TIA 33 (19.5) 63 (28.2)   96 (24.5)
 Time between screenings, minutes (median) 22 19.5   20
 Time from screening to SLP evaluation, h 13 NA   13

NIHSS = National Institutes of Health Stroke Scale; SLP = speech language pathologist; TIA = transient ischemic attack.

Seventy-seven percent (n = 297) of patients passed the dysphagia screening. In Part I, 4 patients failed the medical history section, and an additional 4 failed by not being able to sit properly for the examination. In Part II, which assessed cognitive and motor skills, 79 patients failed: 16 failed due to altered level of consciousness or inability to attend to cues; 24 failed to control saliva, lacked tongue control, or were unable to smile and/or pucker; and 39 failed voice quality. In Part III, which included water tests, 2 patients failed because they did not respond to ice chips in the mouth, 4 did not tolerate ice chips, and 2 did not tolerate water without clearing the throat during and 1 minute afterward (Table 2).

Table 2. Results of dysphagia screening

Screen section Passed Failed Passed Failed Passed Failed
Region 1 Region 2 Combined
Part I History 284 3 104 1 388 4
Part I Patient able to sit properly 281 3 103 1 384 4
Part I Patient level of consciousness: Patient awake for  20 min 269 12 99 4 368 16
Part II Patient level of consciousness: Able to attend to cues 260 9 96 3 356 12
Part II Secretions tongue control: Able to swallow  secretions/handle secretions without drooling 258 2 95 1 353 3
Part II Secretions tongue control: Able to manage saliva  without drooling 256 2 94 1 350 3
Part II Secretions tongue control: Able to move tongue side  to side 254 2 94 0 348 2
Part II Secretions tongue control: Able to smile and pucker 251 3 93 1 344 4
Part II Sounds and cough: Able to make sounds 249 2 93 0 342 2
Part II Vocal quality and speech: Speech is clear, not  slurred/dysarthric 224 25 90 3 314 28
Part II Vocal quality and speech: Voice sounds strong and  clear 221 3 88 2 309 5
Part II Vocal quality and speech: Voice does not sound  wet/weak/hoarse 221 0 87 1 308 1
Part II Vocal quality and speech: Able to say extended "ahh" 221 0 86 1 307 1
Part II Laryngeal movement: A rise in larynx is felt with  swallow 219 2 86 0 305 2
Part III Ice chip test: Patient responds to ice in mouth 217 2 86 0 303 2
Part III Ice chip test: Rise is felt in larynx when patient  swallows 217 0 86 0 303 0
Part III Ice chip test: Tolerates w/o coughing during AND  1 min after 213 4 86 0 299 4
Part III Ice chip test: Tolerates w/o throat clearing during  AND 1 min after 213 0 86 0 299 0
Part III Ice chip test: Voice is clear when saying an extended 213 0 86 0 299 0
Part III Water test: Patient responds to water in mouth 213 0 86 0 299 0
Part III Water test: Rise is felt in larynx when patient  swallows 213 0 86 0 299 0
Part III Water test: Tolerates without coughing during AND  1 min after 211 2 86 0 297 2
Part III Water test: Tolerates without throat clearing during  AND 1 min after 211 0 86 0 297 0
Part III Water test: Voice is clear when saying an extended  "ahh" 211 0 86 0 297 0

A total of 379 subjects received 2 dysphagia screenings for comparison; 13 patients had only one dysphagia screening. Seventy-two percent (n = 272) of patients passed both dysphagia screenings; 22% (n = 83) failed both screens. The reliability of the dysphagia screening across nurses was high, with 93.7% agreement and interrater reliability of Ƙ = 0.83 (Table 3).

Table 3. Interreliability of 2 registered nurse-blinded dysphagia screenings

Swallow evaluation agreement n %
 Both pass 272 71.8
 Both fail 83 21.9
 No agreement 24 6.3
 Total 379  
Measure of reliability Reliability estimate
 Percent agreement 93.7%
 Probability of passing 74.9%
 Expected Agreement 62.4%
 Interrater reliability (kappa) 83.1%

A total of 169 subjects had both a screening and an evaluation completed to determine the validity of the dysphagia screening tool. Among the 97 patients who passed the RN dysphagia screening, 91 also passed the SLP swallow evaluation. Of the 72 patients who failed the RN dysphagia screening, 38 also failed the SLP swallow evaluation (Table 4). The dysphagia screening performed well when compared with the formal SLP swallow evaluation, demonstrating both high sensitivity (86.4%; 95% CI = 73.3-93.6) and high negative predictive value (93.8%; 95% CI = 87.2-97.1). It also demonstrated moderately high specificity (72.8%; 95% CI = 64.4-79.8), with a PPV of 52.8% (95% CI = 45.3-60.2) (Table 4).

Table 4. Validation of registered nurse swallow screening against the speech language pathologist swallow evaluation

Swallow screening result Combined Region 1 and Region 2
Speech evaluation result
Pass* Fail* Total
 Pass 91 6 97
 Fail 34 38 72
 Total 125 44 169
  Validation estimate
Measure of validation Estimate 95% CI lower 95% CI upper
 Sensitivity 86.4% 73.3 93.6
 Specificity 72.8% 64.4 79.8
 PPV 52.8% 45.3 60.2
 NPV 93.8% 87.2 97.1

* "Pass" (low dysphagia risk) = thin liquids; "Fail" (high dysphagia risk) = nothing by mouth or modified liquids. For predictive value: "positive" = dysphagia (fail screen); "negative" = no dysphagia (pass screen).

CI = confidence interval; NPV = negative predictive value; PPV = positive predictive value.

DISCUSSION

The tool demonstrated high sensitivity (86.4%) to accurately detect dysphagia. The dysphagia screening tool had high specificity (72.8%) to identify patients who do not have dysphasia, which minimizes the time spent without oral intake. The low PPV (52.8%) indicates an increased failure rate by the nurse vs the SLP evaluation, providing increased safety for the patient. Patients without dysphagia may be mistakenly identified as positive; however, the patients can then be cleared by the SLP evaluation. Thus, the risk of those with dysphagia receiving fluids or medications is reduced.8,16 This dysphagia screening tool compares favorably with other screening tools for reliability and validity.8 The tool also includes necessary components: identification of possible dysphagia, a simple scoring system, ease of completion for health care workers outside of an SLP, and the ability to rescreen when clinical changes indicate.17,19,20 The screening is a reliable preliminary assessment of a patient’s ability to swallow. The protocol’s validity was measured against the gold standard of an SLP evaluation.20 The tool is easy to learn and quick to perform, taking an average of 10 minutes to complete.

Limitations

There was variation in the elapsed time between the 2 RN dysphagia screenings and between the RN screenings and the SLP evaluation. The demands on the RN’s time due to unit acuity affected the time between RN screenings. The time between RN screenings and SLP evaluation was affected by the SLP’s availability. However, the impact was minimal because the median time between the 2 dysphagia screenings was 20 minutes, and there were 13 hours between the dysphagia screening and the SLP evaluation (Table 1). Blinding between the 2 RNs was done on the honor system because the subinvestigators were not present for all dysphagia screenings. Hospitalists were informed of the study. However, there were subjects who had passed the screening but did not receive a physician order for an SLP evaluation. These subjects were eliminated. Individual outcome data for the occurrence of aspiration pneumonia in patients who passed the screening were not available. Only the aggregate rate of aspiration pneumonia is known.

CONCLUSION

The Dysphagia Screening Tool is reliable and valid and has minimal training requirements. Thousands of patients in both large and smaller acute care settings have been screened in a timely manner by nurses. Although this study was not able to correlate use of the tool with outcomes, the healthcare system overall has low aspiration rates (2%-3%) for patients with acute ischemic strokes.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Medical Institutions work was done: Kaiser Permanente Northern California (KPNC) Medical Centers - Antioch, Oakland, Richmond, Sacramento, San Francisco, San Jose, Santa Clara, South Sacramento, South San Francisco, Walnut Creek, Vacaville, Vallejo, Kaiser Permanente Southern California (KPSC) Medical Centers – Anaheim, Irvine, Baldwin Park, Downey, Fontana, Moreno Valley, Panorama City, Riverside, and West Lost Angeles.

Acknowledgments

The authors would like to thank patients who have suffered a stroke who continue to inspire the work to provide the best care possible. We would like to thank the staff and facilities who participated in the study.

Author Affiliations

1Kaiser Permanente Northern California, Redwood City Medical Center, Redwood City, CA

2Kaiser Permanente Northern California Regional Licensing, Quality and Accreditation, Oakland, CA

3Kaiser Permanente Southern California, Moreno Valley Medical Center, Moreno Valley, CA

4Kaiser Permanente Northern California Division of Research, Oakland, CA

5Kaiser Permanente Southern California, Riverside Medical Center, Riverside, CA

6Kaiser Permanente Northern California, Diablo Service Area, Walnut Creek, CA

Corresponding Author

Melissa M Meighan DNP, MS, RN, CNRN, SCRN, NEA-bc melissa.m.meighan@kp.org

How to Cite this Article

Finnegan BS, Meighan MM, Warren NC, et al. Validation study of Kaiser Permanente bedside dysphagia screening tool in acute stroke patients. Perm J 2020;24:19.230. DOI: https://doi.org/10.7812/TPP/19.230

References

1. Bray BD, Smith CJ, Cloud GC, et al The association between delays in screening for and assessing dysphagia after acute stroke, and the risk of stroke-associated pneumonia. J Neurol Neurosurg Psychiatry 2017 Jan;88(1):25-30. DOI: https://doi.org/10.1136/jnnp-2016-313356

2. Palli C, Fandler S, Doppelhofer K, et al Early dysphagia screening by trained nurses reduces pneumonia rate in stroke patients: a clinical intervention study. Stroke 2017 Sep;48(9):2583-85. DOI: https://doi.org/10.1161/STROKEAHA.117.018157

3. Martino R, Foley N, Bhogal S, Diamant N, Speechley M, Teasell R. Dysphagia after stroke: incidence, diagnosis, and pulmonary complications. Stroke 2005 Dec;36(12):2756-63. DOI: https://doi.org/10.1161/01.STR.0000190056.76543.eb

4. Campbell GB, Carter T, Kring D, Martinez C. Nursing bedside dysphagia screen: is it valid? J Neurosci Nurs 2016 Apr;48(2):75-9. DOI: https://doi.org/10.1097/JNN.0000000000000189

5. Ouyang M, Boaden E, Arima H, et al Dysphagia screening and risks of pneumonia and adverse outcomes after acute stroke: an international multicenter study. Int J Stroke 2020 Feb:206-15. DOI: https://doi.org/10.1177/1747493019858778.

6. Martino R, Beaton D, Diamant NE. Perceptions of psychological issues related to dysphagia differ in acute and chronic patients. Dysphagia 2010 Mar;25(1):26-34. DOI: https://doi.org/10.1007/s00455-009-9225-0

7. Titsworth WL, Abram J, Fullerton A, et al Prospective quality initiative to maximize dysphagia screening reduces hospital-acquired pneumonia prevalence in patients with stroke. Stroke 2013 Nov;44(11):3154-160. DOI: https://doi.org/10.1161/STROKEAHA.111.000204

8. Fedder WN. Review of evidenced-based nursing protocols for dysphagia assessment. Stroke 2017 Apr;48(4):e99-101. DOI: https://doi.org/10.1161/STROKEAHA.116.011738

9. Trapl M, Enderle P, Nowotny M, et al Dysphagia bedside screening for acute-stroke patients: the Gugging Swallowing Screen. Stroke 2007 Nov;38(11):2948-52. DOI: https://doi.org/10.1161/STROKEAHA.107.483933.

10. Poorjavad M, Jalaie S. Systemic review on highly qualified screening tests for swallowing disorders following stroke: validity and reliability issues. J Res Med Sci 2014 Aug;19(8):776-85.

11. Donovan NJ, Daniels SK, Edmiaston J, et al Dysphagia screening: state of the art: invitational conference proceeding from the State-of-the-Art Nursing Symposium, International Stroke Conference 2012. Stroke 2013 Apr;44(4):e24-31. DOI: https://doi.org/10.1161/STR.0b013e3182877f57

12. Smith EE, Kent DM, Bulsara KR, et al Effect of dysphagia screening strategies on clinical outcomes after stroke: a systematic review for the 2018 guidelines for the early management of patients with acute ischemic stroke. Stroke 2018 Mar;49(3):e123-8. DOI: https://doi.org/10.1161/STR.0000000000000159.

13. Powers WJ, Rabinstein AA, Ackerson T, et al 2018 guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2018 Mar;49(3):e46-110. DOI: https://doi.org/10.1161/STR.0000000000000158.

14. Massey R, Jedlicka D. The Massey bedside swallowing screen. J Neurosci Nurs 2002 Oct;34(5):252-53, 257-260. DOI: https://doi.org/10.1097/01376517-200210000-00005

15. DePippo KL, Holas MA, Reding MJ. The Burke dysphagia screening test: validation of its use in patients with stroke. Arch Phys Med Rehabil 1994 Dec;75(12):1284-86. https://doi.org/10.1016/0003-9993(94)90274-7

16. Edmiaston J, Connor LT, Loehr L, Nassief A. Validation of a dysphagia screening tool in acute stroke patients. Am J Crit Care 2010 Jul;19(4):357-64. DOI: https://doi.org/10.4037/ajcc2009961

17. Daniels SK, Anderson JA, Willson PC. Valid items for screening dysphagia risk in patients with stroke: a systematic review. Stroke. 2012 Mar;43(3):892-97. DOI: https://doi.org/10.1161/STROKEAHA.111.640946

18. Gee E, Lancaster E, Meltzer J, Mendelsohn AH, Benharash P. A targeted swallow screen for the detection of postoperative dysphagia. Am Surg 2015 Oct;81(10):979-82. DOI: https://doi.org/10.1177/000313481508101014.

19. Behera A, Read D, Jackson N, Saour B, Alshekhlee D, Mosier AK. A validated swallow screener for dysphagia and aspiration in patients with stroke. J Stroke Cerebrovasc Dis 2018 Jul;27(7):1897-904. DOI: https://doi.org/10.1016/j.jstrokecerebrovasdis.2018.02.037

20. Cummings J, Soomans D, O'Laughlin J, et al Sensitivity and specificity of a nurse dysphagia screen in stroke patients. Medsurg Nurs 2015 Jul-Aug;24(4):219-22, 263.

Keywords: dysphagia, stroke, swallow screen, bedside swallow, speech language pathology, nursing swallow screen

ETOC

Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.

CIRCULATION

2 million page views of TPJ articles in PubMed from a broad international readership.

Indexing

Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, and CrossRef.


                                             

 

 

ISSN 1552-5775 Copyright © 2021 thepermanentejournal.org

All Rights Reserved