Impact of Asynchronous Training on Radiology Learning Curve among Emergency Medicine Residents and Clerkship Students

Impact of Asynchronous Training on Radiology Learning Curve among Emergency Medicine Residents and Clerkship Students

 

Ali Pourmand, MD, MPH, RDMS; Christina Woodward, MD; Hamid Shokoohi, MD, MPH, RDMS;
Jordan B King, PharmD, MS; M Reza Taheri, MD, PhD; Jackson King; Christopher Lawrence

Perm J 2018;22:17-055 [Full Citation]

https://doi.org/10.7812/TPP/17-055
E-pub: 12/18/2017

ABSTRACT

Context: Web-based learning (WBL) modules are effectively used to improve medical education curriculum; however, they have not been evaluated to improve head computed tomography (CT) scan interpretation in an emergency medicine (EM) setting.
Objective: To evaluate the effectiveness of a WBL module to aid identification of cranial structures on CT and to improve ability to distinguish between normal and abnormal findings.
Design: Prospective, before-and-after trial in the Emergency Department of an academic center. Baseline head CT knowledge was assessed via a standardized test containing ten head CT scans, including normal scans and those showing hemorrhagic stroke, trauma, and infection (abscess). All trainees then participated in a WBL intervention. Three weeks later, they were given the same ten CT scans to evaluate in a standardized posttest.
Main Outcome Measures: Improvement in test scores.
Results: A total of 131 EM clerkship students and 32 EM residents were enrolled. Pretest scores correlated with stage of training, with students and first-year residents demonstrating the lowest scores. Overall, there was a significant improvement in percentage of correctly classified CT images after the training intervention from a mean pretest score of 32% ± 12% to posttest score of 67% ± 13% (mean improvement = 35% ± 13%, p < 0.001). Among subsets by training level, all subgroups except first-year residents demonstrated a statistically significant increase in scores after the training.
Conclusion: Incorporating asynchronous WBL modules into EM clerkship and residency curriculum provides early radiographic exposure in their clinical training and can enhance diagnostic head CT scan interpretation.

INTRODUCTION

More than 795,000 people sustain a stroke in the US annually, making it the leading cause of prolonged disability and the third most common cause of death in the country.1 An additional 2.5 million Americans sustain head injuries resulting in Emergency Department (ED) visits, hospitalizations, and deaths.2 In both cases, head computed tomography (CT) scans are frequently used and are an integral part of diagnosis. Emergency medicine (EM) physicians routinely order head CT scans in cases of suspected emergent pathology, and on-site radiologists are not always available, so it is important for EM physicians to know how to interpret the scans.

Large academic medical centers frequently have round-the-clock (“24/7”), on-site radiologists; however, this level of support is not always available. Many community hospitals are not able to provide a trained radiologist on-site who is accessible overnight,3 and even large academic medical centers often have residents performing nighttime preliminary readings without the oversight of an attending physician. As such, teleradiology is becoming increasingly common.4 This model of staffing is potentially more cost-efficient, but CT scans read by teleradiologists may carry a higher misinterpretation rate compared with in-house radiology interpretations.5 Given the potential for differences in image quality or lack of access to helpful supplementary patient data that the teleradiologist is contending with, this finding is understandable. Other potential issues with the use of teleradiology include timelines of scan interpretations, a lack of “finalized” interpretation reports, variability in training background, and the qualifications of the teleradiologist. All together, these challenges highlight the necessity that community EM physicians have a degree of proficiency in reading images on their own.

Prior studies have found asynchronous online learning—a method to deliver didactic contents through an Internet-based platform—to be an effective way of imparting medical knowledge to EM trainees.6,7 For example, in a study of EM clerkship students and residents, the use of a Web-based learning (WBL) module was found to significantly increase the ability of the learner to recognize acute ST-segment elevations in myocardial infarctions on an electrocardiogram.8 However, to our knowledge, this approach has yet to be implemented toward improving EM medical students’ and residents’ knowledge of interpreting head CT scans. The primary objective of the current study was to develop a WBL module and to evaluate the effectiveness of the module at improving the ability of EM clerkship students and EM residents to identify cranial structures (including normal anatomy and abnormal intracranial pathology) on CT and to improve their ability to distinguish between normal and abnormal CT findings.

METHODS

Design and Setting

We conducted a single-arm, before-and-after trial of a WBL intervention aimed at improving interpretation and understanding of head CT scans by medical trainees in an ED setting.

The study was conducted in the ED of an urban academic center, including a stroke center with 24/7 availability of inhospital neurology and neurosurgery services and an annual census of 75,000 patients.

Recruitment

The study was conducted during the 2012-2013 academic year, and all participants were EM trainees. All EM residents (four-year program) and clerkship students in the ED were eligible to participate regardless of prior experience with head CT. All eligible trainees chose to participate.

Web-based Learning Intervention

All participants attended an Internet-delivered, video lecture on head CT. The video was prerecorded and presented by a board-certified neuroradiologist. The focus of the lecture was on interpretation and understanding of head CT scans. The total running time for the video was 13 minutes and 50 seconds, which was broken into 3 main topics. First, a general overview of common head CT indications and basic technique for CT was discussed. Second, normal CT head anatomy was presented and contrasted with abnormal anatomy. Finally, how to detect common aberrant findings was presented. This included epidural hematoma, subdural hematoma, subarachnoid hemorrhage, intraparenchymal hemorrhage, diffuse axonal injury, stroke, abscess, and masses with corpus callosum involvement.

Before viewing this module, baseline head CT knowledge was assessed by administering a standardized pretest to all students and residents, asking each to identify normal head CT scans and common pathology. The pretest contained ten head CT scans and included normal scans as well as those relating to intracranial hemorrhage, trauma, and infection (specifically, abscess). All students then participated in the WBL module. They were given the same ten CT scans to evaluate in a standardized posttest three weeks later. All subjects were asked for their voluntary participation in the study. All participants were made aware of the posttest at the time of verbal consent. Neither test performance nor test questions were discussed with the participants. The test questions used static images, and the subjects had to write the correct answers. The test was scored on an ordinal scale with one point given for each correctly read scan. All participants completed both the pretests and posttests. This study protocol was reviewed and approved by the institutional review board of George Washington University School of Medicine.

Outcome Measures

The primary outcome was the improvement in test score before and after the WBL intervention. Secondary outcomes included absolute test score after the intervention and subgroup analysis of benefit of intervention based on year of medical training.

Statistical Analysis

Descriptive statistics, including means and standard deviations and medians and interquartile ranges (25th to 75th percentiles), were performed overall and by stage of training (eg, medical student, postgraduate year [PGY]-1, PGY-2, etc) for pre- and posttest scores. A repeated-measures analysis of test scores was performed using a Wilcoxon signed-rank test to assess changes in individual test score from before and after the multimedia training presentation. Analysis of covariance was used to assess the impact of the participant’s stage of training on the achieved benefit of the intervention, adjusting for baseline test scores. Stage of training was assessed both as a dichotomous variable (medical student vs EM resident) and as a categorical variable for each year of postgraduate residency training. All analysis was performed using statistical software (Stata 14.1, Stata Corp, College Station, TX).

RESULTS

Data were collected during the academic year of 2012-2013. During this year, 131 EM clerkship students and 32 EM residents (8 PGY-1, 9 PGY-2, 8 PGY-3, and 7 PGY-4) were enrolled and participated in both pre- and posttests.

Descriptive statistics for results of the pretest, posttest, and change from pre- to posttest scores for participants, aggregated both overall and within year of medical training, are provided in Table 1. Overall, the percentage of correctly diagnosed CT images before the WBL intervention was 32% ± 12% (mean ± standard deviation). The absolute range of correctly diagnosed CT images in the pretest was 1 (10% correct) to 6 (60% correct). There was a weak yet significant correlation between pretest scores and stage of training (r = 0.29, p = 0.002). The medical clerkship students and PGY-1 EM residents had the lowest percentage of correctly diagnosed CT images (31% ± 12%), and PGY-3 and PGY-4 EM residents had the highest percentage (44% ± 15%). The PGY-4 EM residents were the only individual group to have a median score of at least 5 (50% correct) in the pretest period.

The percentage of correctly diagnosed CT images after attending the WBL intervention increased to 67% ± 13%. There was significant variation between stage of medical education and mean posttest score, with medical students (69% ± 11% of images correctly classified) and PGY-4 residents (66% ± 21%) achieving the highest scores. Among the EM residents, there was a graded effect by year, with posttest score performance improving with each year of training. No participant obtained a perfect score in the posttest period.

All 131 medical students and 26 (81.3%) of the EM residents demonstrated improvement in scores from pretest to posttest. The overall mean change in pre- and posttest images correctly classified was 35% ± 13%. All stages of medical education subgroups except PGY-1 demonstrated a statistically significant increase in score after the training intervention. The degree of improvement was most prominent among medical students compared with EM residents (Figures 1 and 2).

The analysis of covariance demonstrated a significant association between status as a medical student vs a resident and an improvement in test scores while adjusting for baseline head CT knowledge (Table 2 and Figure 3). Medical students achieved a 1.7-point (17%) increase in posttest scores compared with EM residents while analysis controlled for baseline knowledge. Significant variation was also observed when each year of medical education was examined as a categorical variable.

Impact of Asynchronous Training on Radiology Learning Curve among Emergency Medicine Residents and Clerkship StudentsImpact of Asynchronous Training on Radiology Learning Curve among Emergency Medicine Residents and Clerkship Students

Impact of Asynchronous Training on Radiology Learning Curve among Emergency Medicine Residents and Clerkship Students

Impact of Asynchronous Training on Radiology Learning Curve among Emergency Medicine Residents and Clerkship Students

Impact of Asynchronous Training on Radiology Learning Curve among Emergency Medicine Residents and Clerkship Students

DISCUSSION

In this study, we demonstrated that a WBL module could be used to improve knowledge of head CT interpretation among medical trainees. All individual trainee groups evaluated except PGY-1 residents experienced improvement in head CT test scores after attendance of the WBL module; however, variation existed in the absolute benefit obtained. Medical students, who have less baseline knowledge of head CT, may be more likely to benefit from the additional instruction, but this was not clearly observed in the PGY-1 residents.

The results of this study support WBL in medical education, a concept that has been introduced in a number of past studies.9-12 Asynchronous online learning is a relatively new, yet increasingly popular, instructional tool in the field of EM, as evidenced by the growing body of podcasts, e-learning, and blogs in the EM community.13 One study found that knowledge acquisition among EM residents viewing lectures in an online asynchronous structure was equal to or better than those who attended traditional core content lectures.6 Another study demonstrated improved recognition of ST-elevation myocardial infarction in both EM clerkship students and EM residents after viewing a WBL module that demonstrated acute electrocardiographic changes in this setting.8

The benefits of online asynchronous learning are numerous. After a traditional face-to-face lecture, only 42% of key points can be recalled. Within 1 week, this percentage drops to 20%.14 Given natural limitations to the human memory and its ability to focus, a WBL format that allows the learner to pause, rewind, and rewatch the lesson as often as desired may be the key to achieving maximal knowledge retention. Online lessons allow learners ideal flexibility, giving them the ability to consume lectures at the time and location most convenient to them.15 Additionally, WBL modules are easily updatable, allowing the instructor to alter learning points or add information as new studies emerge.

It is our opinion that a WBL module such as that used in the present study is a particularly powerful tool to assist the EM learner in mastering neuroimaging interpretation, given the easy accessibility and allowance for repetition that it provides. An ED requires 24/7 coverage, necessitating a variable shift schedule for all learners, including students and residents, that inevitably leads to numerous missed academic core lectures. A WBL tool enables all learners easy access to instructional information that can be viewed at any time of day, removing the burden of scheduling from the teacher. The present study was conducted in EM clerkship students and EM residents, and demonstrated a significant learning benefit on interpretation of head CT scans after the use of an online module. Attending physicians have similar scheduling restraints and thus could capitalize on the same benefits that an online module offers; we hypothesize that a WBL module would also likely be helpful in EM physician education on neuroimaging interpretation.

In recent years, the idea of a “flipped-classroom model” has been posed, where digital lectures are consumed at home, allowing classroom time to be used for interactive, higher-order teaching such as case-based problems, small group discussion, or simulation.16,17 A WBL module such as ours could be easily substituted for a traditional face-to-face core lecture in a flipped-classroom model. Although medical education is trending toward increased use of technologic teaching tools, the era of the flipped-classroom model has not yet come to fruition.

Modern-day teaching requires a successful instructor to provide a variety of teaching methods, under the assumption that each learner will benefit from a different style of instruction.18 Online modules can be tailored to meet individual needs by including a variety of teaching tools for the learner to select from, such as Web-based lectures, interactive modules, periodic quizzes, and optional hyperlinks. There may come a day in the future when WBL takes the place of classroom lectures altogether, but currently it is, at the very least, a helpful teaching adjunct.

This study has several limitations. It was conducted in EM clerkship students and EM residents in a single academic medical center, which may limit wider applicability of our results. Our teaching tool was developed by one of our on-site specialists and has not been formally validated for neuroradiology education. In addition, the lack of a randomized control group limits the interpretation of these results. The findings of this study should be further validated by a larger trial with randomization to an active WBL intervention arm or usual training/control arm. As a pre-post study without a control group, this study can provide only limited evidence about the effect of the intervention.

CONCLUSION

Introducing asynchronous WBL modules into EM clerkship curriculum offers early radiographic exposure and can improve neuroimaging interpretation. In an era of exciting technologic innovations and an emphasis on providing variable teaching methods, asynchronous online learning can be a valuable teaching adjunct.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Acknowledgments

Kathleen Louden, ELS, of Louden Health Communications provided editorial assistance.

How to Cite this Article

Pourmand A, Woodward C, Shokoohi H, et al. Impact of asynchronous training on radiology learning curve among emergency medicine residents and clerkship students. Perm J 2018;22:17-055. DOI: https://doi.org/10.7812/TPP/17-055

References
1.    Writing Group Members, Mozaffarian D, Benjamin EJ, Go AS, et al; American Heart Association Statistics Committee; Stroke Statistics Subcommittee. Heart disease and stroke statistics—2016 update: A report from the American Heart Association. Circulation 2016 Jan 26;133(4):e38-360. DOI: https://doi.org/10.1161/CIR.0000000000000350. Erratum in: Circulation 2016 Apr 12;133(15):e599. DOI: https://doi.org/10.1161/CIR.0000000000000409.
    2.    Centers for Disease Control and Prevention; Frieden TR, Houry D, Baldwin G. The report to Congress on traumatic brain injury in the United States: Epidemiology and rehabilitation [Internet]. Atlanta, GA: Centers for Disease Control and Prevention; 2014 [cited 2015 Nov 1]. Available from: www.cdc.gov/traumaticbraininjury/pdf/tbi_report_to_congress_epi_and_rehab-a.pdf.
    3.    Saketkhoo DD, Bhargavan M, Sunshine JH, Forman HP. Emergency department image interpretation services at private community hospitals. Radiology 2004 Apr;231(1):190-7. DOI: https://doi.org/10.1148/radiol.2311030307.
    4.    Kammerer FJ, Hammon M, Schlechtweg PM, Uder M, Schwab SA. A web based cross-platform application for teleconsultation in radiology. J Telemed Telecare 2015 Sep;21(6):355-63. DOI: https://doi.org/10.1177/1357633x15575237.
    5.    Platts-Mills TF, Hendey GW, Ferguson B. Teleradiology interpretations of emergency department computed tomography scans. J Emerg Med 2010 Feb;38(2):188-95. DOI: https://doi.org/10.1016/j.jemermed.2008.01.015.
    6.    Pourmand A, Lucas R, Nouraie M. Asynchronous web-based learning, a practical method to enhance teaching in emergency medicine. Telemed J E Health 2013 Mar;19(3):169-72. DOI: https://doi.org/10.1089/tmj.2012.0119.
    7.    von Sass PF, Scheckenbach K, Wagenmann M, Klenzer T, Schipper J, Chaker A. Taking a fresh look at the skull base in otorhinolaryngology with web-based simulation: Student’s Interactive Skull-Base Trainer (SISTer). JAMA Otolaryngol Head Neck Surg 2015 Feb;141(2):154-9. DOI: https://doi.org/10.1001/jamaoto.2014.3041.
    8.    Pourmand A, Tanski M, Davis S, Shokoohi H, Lucas R, Zaver F. Educational technology improves ECG interpretation of acute myocardial infarction among medical students and emergency medicine residents. West J Emerg Med 2015 Jan;16(1):133-7. DOI: https://doi.org/10.5811/westjem.2014.12.23706.
    9.    Cook DA, Garside S, Levinson AJ, Dupras DM, Montori VM. What do we mean by web-based learning? A systematic review of the variability of interventions. Med Educ 2010 Aug;44(8):765-74. DOI: https://doi.org/10.1111/j.1365-2923.2010.03723.x.
    10.    Jayakumar N, Brunckhorst O, Dasgupta P, Khan MS, Ahmed K. E-learning in surgical education: A systematic review. J Surg Educ 2015 Nov-Dec;72(6):1145-57. DOI: https://doi.org/10.1016/j.jsurg.2015.05.008.
    11.    Tarpada SP, Morris MT, Burton DA. E-learning in orthopedic surgery training: A systematic review. J Orthop 2016 Sep 21;13(4):425-30. DOI: https://doi.org/10.1016/j.jor.2016.09.004.
    12.    Vasilopoulos T, Chau DF, Bensalem-Owen M, Cibula JE, Fahy BG. Prior podcast experience moderates improvement in electroencephalography evaluation after educational podcast module. Anesth Analg 2015 Sep;121(3):791-7. DOI: https://doi.org/10.1213/ane.0000000000000681.
    13.    Gosbell A, Killen A, McCarthy S. Australasian College for Emergency Medicine, Emergency Medicine Australasia and social media. Emerg Med Australas 2012 Aug;24(4):463-4. DOI: https://doi.org/10.1111/j.1742-6723.2012.01579.x.
    14.    Swanwick T. Understanding medical education: Evidence, theory and practice. 1st ed. Chichester, West Sussex, UK: John Wiley & Sons; 2010. p 17.
    15.    Cook DA. Web-based learning: Pros, cons and controversies. Clin Med (Lond) 2007 Jan-Feb;7(1):37-42. DOI: https://doi.org/10.7861/clinmedicine.7-1-37.
    16.    Prober CG, Heath C. Lecture halls without lectures—a proposal for medical education. N Engl J Med 2012 May 3;366(18):1657-9. DOI: https://doi.org/10.1056/nejmp1202451.
    17.    Sherbino J, Chan T, Schiff K. The reverse classroom: Lectures on your own and homework with faculty. CJEM 2013 May;15(3):178-80. DOI: https://doi.org/10.2310/8000.2013.130996.
    18.    Vaughn L, Baker R. Teaching in the medical setting: Balancing teaching styles, learning styles and teaching methods. Med Teach 2001 Oct;23(6):610-12. DOI: https://doi.org/10.1080/01421590120091000.

The Permanente Journal

Sponsored by the eight Permanente Medical Groups, The Permanente Journal advances knowledge in scientific research, clinical medicine, and innovative health care delivery.

Reprint Permissions

The Permanente Journal welcomes requests for reprints and reproduction. Use of any and all material published in The Permanente Journal is copyrighted and protected.

The Permanente Press

The Permanente Press publishes The Permanente Journal and books related to healthcare. Journal subscriptions are entered for the calendar year. Advance payment in US dollars is required.


ISSN 1552-5775 Copyright © 2018 thepermanentejournal.org.

The Permanente Press. All Rights Reserved.