An Evaluation of the SCORE Program: A Novel Research and Mentoring Program for Medical Students in Obstetrics/Gynecology and Otolaryngology



 

Kellie Corcoran, MD, MPH1; Miranda Ritterman Weintraub, PhD, MPH2; Isabella Silvestre1;
Reshma Varghese1; Jonathan Liang, MD3; Eve Zaritsky, MD1

Perm J 2020;24:19.153 [Full Citation]

https://doi.org/10.7812/TPP/19.153
E-pub: 04/16/2020

Abstract

Background: There has been a steady decrease in the number of physician-scientists and a lack of diversity and inclusion of underrepresented minorities (URMs) in medicine.

Objective: To assess the research productivity, interest, and experience of medical students, including URMs, and resident and faculty mentors of the Kaiser Permanente Oakland Medical Center’s 8-week, intensive, mentored Summer Clinical Otolaryngology and Obstetrics/Gynecology Research (SCORE) Program for second-year medical students.

Methods: A database of SCORE Program research projects was generated from 2016, when the program was launched, through 2018. SCORE Program students and faculty completed a brief, mixed-methods, anonymous exit survey that captured respondents’ experiences, perceived program strengths, and opportunities for improvement. The number of peer-reviewed manuscripts produced were counted.

Results: A total of 16 SCORE Program students (50% female and 38% URMs) between 2016 and 2018 and 8 residents and 8 faculty members in 2018 completed a brief, mixed-methods, anonymous exit survey that captured respondents’ experiences, perceived program strengths, and opportunities for improvement. The medical students coauthored 12 published peer-reviewed articles, 25 abstracts or national posters, and 1 opinion editorial. According to the program’s annual exit survey, 87% of students reported an increased interest in pursuing research, 93% had an increased interest in their respective specialties, and 93% believed they had positive mentorship experiences. Similarly, faculty and residents enjoyed mentoring students, thought that students positively contributed to their projects, and would support the SCORE Program in the future.

Conclusion: Structured research and mentoring experiences, such as the SCORE program, may encourage students, including URMs, to pursue research throughout their careers in addition to adding research to their curriculum vitae to strengthen their residency applications.

Introduction

The number of physician-scientists has been steadily decreasing for decades, with more physicians opting to pursue purely clinical careers.1 Physician-scientists, through their direct patient interactions and experience in clinical settings, are ideally positioned to identify clinically driven research topics and translate research findings into direct patient care. In addition, despite efforts to improve representation of minority persons in medicine, only 7% of full-time academic faculty within medical schools based in the US identify as an underrepresented minority (URM).2 Training in research and clinical care to a racially and ethnically and socioeconomically diverse trainee population will not only help physician-scientists deliver evidence-based, data-driven care to patients,3 helping to produce a population of physician-scientists capable of identifying a diversity of areas in need of further investigation, but also build a physician-scientist population that better reflects the increasingly diverse patient population.4

In response to the decreasing number of physician-scientists and given the small number of participants in medical and doctoral programs, the medical education system is unable to meet the physician-scientist pool demand.5 There has been a push to incorporate research opportunities into residency training settings. Although these programs train participants in research skills and critical thinking, they are often underfunded and occur too late in training to influence physicians to further pursue research.6,7 In addition, although efforts have been made to improve diversity in academic medicine, including the establishment of mentorship programs and the targeting of undergraduate (and high school) students, these research programs do not take into consideration the diversity of their trainees.8-10

In an effort to reach physicians earlier in their careers, several research programs now exist specifically to train medical students. Medical schools that have implemented mandatory research in the students’ third year found that a greater number of graduating students were interested in research if they had positive experiences with mentors or success in publication.11-13 There are also several research programs geared toward medical students in the summer between their first and second years of school. Students participating in these highly structured and supportive programs reported greater interest in research and were more likely to conduct research after medical school.12-14

Kaiser Permanente Oakland Medical Center initiated the 8-week Summer Clinical Otolaryngology and Obstetrics/Gynecology Research (SCORE) Program in 2016 to provide early clinical exposure and research training to second-year medical students. The SCORE Program is unique in that it attempts to provide early clinical research exposure as well as serve as a pipeline for URMs in medicine. The Association of American Medical Colleges defines URMs as “racial and ethnic minorities in a designated health profession discipline relative to the percentage of that racial or ethnic group in the total population.”15

 The aim of this initial evaluation of the SCORE Program students and resident and faculty mentors was to identify the scholarly activity productivity of the program, assess the program’s impact on student interest in pursuing residencies and research in otolaryngology as well as obstetrics/gynecology (OB/GYN), identify areas for improvement, and track the number of applicants to the program. In addition, perceived resident and faculty benefits were assessed.

Methods

The SCORE Program is an 8-week, intensive summer program at the Kaiser Permanente Oakland Medical Facility for 6 second-year medical students. In its first year (2016), the SCORE Program accepted 4 second-year medical students to participate in research in otolaryngology. In 2017, SCORE expanded to include 2 students in OB/GYN research. The program continued with 6 students in 2018. Invitations to apply online were sent to medical schools across the US in the winter via graduate medical offices, and participants were selected in the spring.

During the program, students were matched with faculty and resident mentors on the basis of student research interest. The students assisted with ongoing research projects at various stages, supporting resident and faculty scholarly activity efforts via literature searches, data collection, data analysis, medical record review, abstract and manuscript writing, and/or oral or poster presentation development and delivery. to provide a diverse range of experiences, students in OB/GYN were encouraged to participate in multiple projects. In otolaryngology, students were assigned to 1 research project and mentor.

Throughout the program, approximately 75% of student time was dedicated to participation in research, whereas the remainder was devoted to clinical observation and didactic learning. Students observed a variety of clinics and operations within their department to better understand different career paths available within the specialty. In addition, students attended resident didactics, grand rounds, and morbidity and mortality conferences. Faculty and resident mentors provided weekly guidance and feedback, and students attended a weekly lunchtime lecture series on research methods and professional development. Guest speakers covered research-related topics, including library database searching, patient safety and quality improvement, research methods, biostatistics, and epidemiology, as well as topics focused on professional development, including health disparities, introduction to integrated health care, and a resident panel designed to answer student questions about life as a resident.

At the end of the 8-week program, students gave an oral presentation to the key SCORE Program faculty, residents, and students on their primary research project. The students completed a brief mixed-methods anonymous exit survey before program departure (2016-2018, see Supplementary Material Sidebar: Kaiser Permanente SCORE Exit Survey for Students— Anonymous Survey, available at: www.thepermanentejournal.org/files/2020/19.153-Supp). In 2018, a survey was also created for key faculty and residents (see Supplementary Material Sidebar: Kaiser Permanente SCORE Exit Survey for Residents and Faculty, available at: www.thepermanentejournal.org/files/2020/19.153-Supp). The surveys contained a combination of multiple choice and open-ended questions that evaluated the SCORE Program experience and the program’s strengths and opportunities for improvement.

This research project was deemed exempt by the Kaiser Permanente Northern California Research Determination Office. Data analyses were conducted using Microsoft excel (Microsoft Corp, Redmond, WA), or Stata software (StataCorp, College Station, TX). Quantitative data were aggregated and analyzed using descriptive statistics.

Results

From 2016 to 2018, 16 students participated in the SCORE Program. In 2016, fewer than 20 students applied for 4 slots, more than 70 applied for 6 positions in 2017, and nearly 90 applied for 6 positions in 2018. More than half of the students selected were women (50%), and more than one-third were URMs (37.5%) (Table 1). Most students were from out-of-state medical schools (75%).

The SCORE Program students have thus far been coauthors on a total of 25 posters or presentations at national conferences, 12 publications in peer-reviewed journals, and 1 opinion editorial (Table 2). Thus far, 63% of past SCORE Program students have published articles (2016 cohort: 100%, 2017: 83%, and 2018: 16%) and 75% have been involved in presentations or abstracts. Of those who have published articles, 75% were first authors. The impact factors of the 12 publications ranged from 1.13 to 2.55 (median, 2.442).

Fifteen of the students completed the exit survey between 2016 and 2018, and all 15 reported that they were more likely to pursue clinical research in the future because of participation in the program, believed that the SCORE Program had increased their interest in a particular specialty, and reported that their faculty mentor was available and provided good direction, teaching, and mentorship (Table 3). On the basis of open-ended questions, the students reported on the program’s strengths, including the diverse range of research and clinical experiences and the training it provided, opportunities to interact closely with residents, and valuable mentorship. Overwhelmingly, students reported on the excellent mentorship, shadowing opportunities in their respective specialty, and engaging research experiences (see Sidebar: SCORE Program student comments representing the range of responses summarizing their research experience and providing feedback from the exit survey).

In 2018, an exit survey was administered to the attending faculty and residents. Eight of the 16 residents (50%), 6 of the 16 attending faculty members (37.5%), and both program managers completed the exit survey (Table 4). All faculty, resident, and manager respondents enjoyed supporting the SCORE Program and agreed that they had a good understanding of mentorship expectations and were able to effectively engage with students during their clinical and research experiences. Interestingly, 81% of respondents strongly believed that SCORE Program students were able to contribute to their research in a meaningful and productive way; the remaining 19% agreed or somewhat agreed with this statement. All residents and attending faculty who responded were interested in being involved in the program in the future and provided supportive comments for the program (see Sidebar: Faculty and resident feedback regarding SCORE Program).

Conclusion

The SCORE Program exposed second-year medical students, including URMs, to a structured clinical research program, providing medical students with clinical and research skills training and mentorship, while providing residents and attending faculty with support to advance their research endeavors. This dual support of student mentees and resident and faculty mentors resulted in numerous scholarly products. These findings complement the increasing body of research indicating that early exposure to mentored clinical research during medical school increases interest in, knowledge of, and exposure to specialty-specific research.12-14 Furthermore, the program’s focus on recruiting and mentoring URMs is in alignment with the nationwide goal of improving diversity and inclusion in medicine. Lastly, the curriculum vitae of the SCORE Program students was more robust on their completion of the program, which could strengthen their residency applications.

19 153 tables

Throughout the first 3 years of the SCORE Program, students have reported having positive experiences. All student participants responded that they were more likely to pursue clinical research in the future. In addition, all students ended the 8-week program with an increased interest in their specialty as well as a greater understanding of clinical clerkships and residency. These findings corroborate those of prior studies,16,17 which have found that early specialty exposure, including through mentorship and experience with patients, increases interest in a particular specialty as well as clinical competence.

The applicant pool increased from fewer than 20 applicants in 2016 for 4 positions to nearly 90 applicants in 2018 for 6 positions as the program became more well known. SCORE Program students in the first 3 years of the program came from across the US, with most being outside of California. This finding indicates a desire for structured summer research programs from medical students across the country. Furthermore, interest has been expressed by other departments for program expansion but is limited at this time by funding.

Although the SCORE Program has received positive feedback from students, residents, and faculty, areas for improvement were documented. Although the attending faculty and residents enjoyed hosting students overall, some found it challenging to incorporate the students into a clinical setting. Many physicians and hospital staff not directly mentoring the students were unfamiliar with the SCORE Program despite programmatic emails that were sent.

The Kaiser Permanente SCORE Program is still in its nascent state; therefore, there has not been the opportunity for long-term student follow-up. Future evaluations of this program could investigate the long-term impact it has on prior participants’ careers and interest in research during and beyond residency. In addition, because prior work11 suggests that interest in research is the biggest indicator of engaging in future research efforts, there may be a bias in who applies to the program, which may be something measurable using a pre-SCORE Program survey designed to gauge students’ interest in research at baseline.

Although the long-term impact of the Kaiser Permanente SCORE Program has not been examined, past studies of similar programs have found that early research experiences, strong mentorship, and publication success are all indicators of participation in future research, especially if individuals have a preexisting interest in research.11-13 In addition, mentorship for URMs alone increases their competitiveness when applying to residency programs.17 On the basis of the follow-up surveys, each SCORE student expressed an increased interest in incorporating research into their future careers. We believe that the Kaiser Permanente SCORE Program and similar experiences that expose medical students to research early in their careers may help reverse the downward trend in the number of physician-scientists. Furthermore, this strategy to increase the proportion of URMs in medicine and research may contribute to improvements not only in diversity and inclusion efforts in medical practice but also in better serving our patient community.

19 153 feedback

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Acknowledgments

Thank you to all the SCORE Program students who participated in the program, Kaiser Permanente Graduate Medical Education for financially supporting the program, and Antoinette Niblett, our administrator of the program.

Laura King, ELS, performed a primary copy edit.

Author Affiliations

1 Department of Obstetrics and Gynecology, Kaiser Permanente, Oakland, CA

2 Department of Graduate Medical Education, Kaiser Permanente, Oakland, CA

3 Department of Head and Neck Surgery, Kaiser Permanente, Oakland, CA

Corresponding Author

Eve Zaritsky, MD ()

Author Contributions

Kellie Corcoran, MD, MPH, participated in the acquisition and analysis of data, drafting, and submitting of the final manuscript. Miranda Weintraub, PhD, participated in drafting and submitting of the final manuscript. Isabella Silvestre participated in drafting and submission of the final manuscript. Reshma Varghese participated in drafting of the final manuscript. Eve Zaritsky, MD, and Jonathan Liang, MD, participated in study design, acquisition and analysis of data, and drafting of the final manuscript. All authors have given final approval to this manuscript.

How to Cite this Article

Corcoran K, Weintraub MR, Silvestre I, Varghese R, Liang J, Zaritsky E. An evaluation of the SCORE Program: A novel research and mentoring program for medical students in obstetrics/gynecology and otolaryngology. Perm J 2020;24:19.153. DOI: https://doi.org/10.7812/TPP/19.153

References
1.    Schafer AI. The vanishing physician-scientist? Transl Res 2010 Jan;155(1):1-2. DOI: https://doi.org/10.1016/j.trsl.2009.09.006 PMID:20004354
    2.    Diversity in medical education: Facts & figures 2016. Washington, DC: Association of American Medical Colleges; 2016
    3.    Harding CV, Akabas MH, Andersen OS. History and outcomes of 50 years of physician-scientist training in medical scientist training programs. Acad Med 2017 Oct;92(10):1390-8. DOI: https://doi.org/10.1097/ACM.0000000000001779 PMID:28658019
    4.    Palepu A, Carr PL, Friedman RH, Ash AS, Moskowitz MA. Specialty choices, compensation, and career satisfaction of underrepresented minority faculty in academic medicine. Acad Med 2000 Feb;75(2):157-60. DOI: https://doi.org/10.1097/00001888-200002000-00014 PMID:10693848
    5.    Feldman AM. The National Institutes of Health Physician-Scientist Workforce Working Group report: A roadmap for preserving the physician-scientist. Clin Transl Sci 2014 Aug;7(4):289-90. DOI: https://doi.org/10.1111/cts.12209 PMID:25123835
    6.    Ercan-Fang NG, Rockey DC, Dine CJ, Chaudhry S, Arayssi T. Resident research experiences in internal medicine residency programs—A nationwide survey. Am J Med 2017 Dec;130(12):1470-1476.e3. DOI: https://doi.org/10.1016/j.amjmed.2017.08.033 PMID:28919025
    7.    Villwock JA, Hamill CS, Nicholas BD, Ryan JT. Otolaryngology residency program research resources and scholarly productivity. Otolaryngol Head Neck Surg 2017 Jun;156(6):1119-23. DOI: https://doi.org/10.1177/0194599817704396 PMID:28419807
    8.    Mahoney MR, Wilson E, Odom KL, Flowers L, Adler SR. Minority faculty voices on diversity in academic medicine: Perspectives from one school. Acad Med 2008 Aug;83(8):781-6. DOI: https://doi.org/10.1097/ACM.0b013e31817ec002 PMID:18667896
    9.    Merchant JL, Omary MB. Underrepresentation of underrepresented minorities in academic medicine: The need to enhance the pipeline and the pipe. Gastroenterology 2010 Jan;138(1):19-26.e1. DOI: https://doi.org/10.1053/j.gastro.2009.11.017 PMID:19944787
    10.    Wells RG. Building a better pipeline: The case for undergraduates in gastrointestinal research. Gastroenterology 2007 Sep;133(3):740-1. DOI: https://doi.org/10.1053/j.gastro.2007.07.036 PMID:17854586
    11.    Weaver AN, McCaw TR, Fifolt M, Hites L, Lorenz RG. Impact of elective versus required medical school research experiences on career outcomes. J Investig Med 2017 Jun;65(5):942-8. DOI: https://doi.org/10.1136/jim-2016-000352 PMID:28270407
    12.    Zier K, Wyatt C, Muller D. An innovative portfolio of research training programs for medical students. Immunol Res 2012 Dec;54(1-3):286-91. DOI: https://doi.org/10.1007/s12026-012-8310-x PMID:22418729
    13.    Chang Y, Ramnanan CJ. A review of literature on medical students and scholarly research: Experiences, attitudes, and outcomes. Acad Med 2015 Aug;90(8):1162-73. DOI: https://doi.org/10.1097/ACM.0000000000000702 PMID:25853690
    14.    Solomon SS, Tom SC, Pichert J, Wasserman D, Powers AC. Impact of medical student research in the development of physician-scientists. J Investig Med 2003 May;51(3):149-56. DOI: https://doi.org/10.1136/jim-51-03-17 PMID:12769197
    15.    Frequently asked questions and answers about the new underrepresented in medicine definition. Washington, DC: Association of American Medical Colleges; 2003. p. 1
    16.    Compton MT, Frank E, Elon L, Carrera J. Changes in U.S. medical students’ specialty interests over the course of medical school. J Gen Intern Med 2008 Jul;23(7):1095-100. DOI: https://doi.org/10.1007/s11606-008-0579-z PMID:18612751
    17.    Yang Y, Li J, Wu X, et al. Factors influencing subspecialty choice among medical students: A systematic review and meta-analysis. BMJ Open 2019 Mar 7;9(3):e022097. DOI: https://doi.org/10.1136/bmjopen-2018-022097 PMID:30850399
    18.    Ramanan RA, Taylor WC, Davis RB, Phillips RS. Mentoring matters. Mentoring and career preparation in internal medicine residency training. J Gen Intern Med 2006 Apr;21(4):340-5. DOI: https://doi.org/10.1111/j.1525-1497.2006.00346.x PMID:16686809

Keywords: medical education, medical student, mentor, physician-scientist, research, underrepresented in medicine

 

Kaiser Permanente SCORE Exit Survey for Students—Anonymous

Thank you in advance for taking a few moments to reflect on the SCORE Program. Your responses to these questions will be filed and reviewed anonymously and may inform how to make improvements to the program.

 

1. Program (select one)

a. SCORE-Oto (Otolaryngology–Head & Neck Surgery)

b. SCORE-Ob (Obstetrics and Gynecology)

 

2. The SCORE Program improved my understanding of clinical research (select one answer).

a. Strongly agree

b. Agree

c. Neutral

d. Disagree

e. Strongly disagree

 

3. I was able to participate in clinical research project(s) in a meaningful and productive way (select one answer).

a. Strongly agree

b. Agree

c. Neutral

d. Disagree

e. Strongly disagree

 

4. The SCORE Program increased my interest in the specialty (Otolaryngology–Head & Neck Surgery or OB/GYN) (select one answer).

a. Strongly agree

b. Agree

c. Neutral

d. Disagree

e. Strongly disagree

 

5. I am more likely to pursue a residency in the specialty (Otolaryngology–Head & Neck Surgery or OB/GYN) because of my participation in the program (select one answer).

a. Strongly agree

b. Agree

c. Neutral

d. Disagree

e. Strongly disagree

 

6. I am more likely to pursue clinical research in the future because of my participation in the program (select one answer).

a. Strongly agree

b. Agree

c. Neutral

d. Disagree

e. Strongly disagree

 

7. I had good understanding of my role in the research project(s) (select one answer).

a. Strongly agree

b. Agree

c. Neutral

d. Disagree

e. Strongly disagree

8. My faculty mentor was available and provided good direction, teaching, and mentorship (select one answer).

a. Strongly agree

b. Agree

c. Neutral

d. Disagree

e. Strongly disagree

 

9. The amount of time dedicated to clinical activities (OR, clinic) was (select one answer):

a. Just right

b. Not enough

c. Too much

 

10. The weekly didactic lectures were informative (select one answer).

a. Strongly agree

b. Agree

c. Neutral

d. Disagree

e. Strongly disagree

 

11. How likely are you to recommend the SCORE Program to other medical students (select one answer)?

a. Very likely

b. Likely

c. Neutral

d. Unlikely

e. Very unlikely

 

12. Please rank the following factors in your decision to participate in the SCORE Program. (1 = most important, 5 = least important):

a. Location

b. Monetary stipend

c. Interest in pursuing the specialty (Otolaryngology–Head & Neck Surgery, OB/GYN)

d. Interest in pursuing clinical research

e. Interest in career at Kaiser Permanente

 

13. Didactic/Lecture Series

a. Which was your favorite topic/lecture?

b. Which was your least favorite topic/lecture?

c. List a topic you would have liked to have included in your didactics.

 

14. Describe 3 strengths of the SCORE Program.

15. Describe 3 improvements that can be made to the SCORE Program.

16. Any other comments/suggestions regarding the SCORE Program?

 

Thank you.

 

Kaiser Permanente SCORE Exit Survey for Residents and Faculty

1. Please check which box describes you

Resident              Faculty/Attending      Other

2. Please select your specialty

Head and Neck Surgery         Obstetrics and Gynecology       Other

For the following questions, please select one answer.

During the course of the SCORE Program,

3. I had a good understanding of the purpose, structure and mentorship expectations.

Strongly Disagree Disagree Somewhat Disagree Somewhat Agree Agree Strongly Agree

4. I felt that I was able to effectively engage with the SCORE students during their clinical, lecture and/or research experiences.

Strongly Disagree Disagree Somewhat Disagree Slightly Agree Agree Strongly Agree

5. SCORE students were able to contribute to my research project(s) in a meaningful and productive way.

Strongly Disagree Disagree Somewhat Disagree Slightly Agree Agree

Strongly Agree

6. I enjoyed supporting the SCORE Program.

Strongly Disagree Disagree Somewhat Disagree Somewhat Agree Agree Strongly Agree

7. I would like to be involved with the SCORE Program in the future.

Yes                      No

8. What do you think is most valuable about the SCORE Program?

9. Please provide any additional comments you would like to share.

ETOC

Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.

CIRCULATION

2 million page views of TPJ articles in PubMed from a broad international readership.

Indexing

Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, CrossRef, and SciVerse/Scopus.


                                             

 

 

ISSN 1552-5767 Copyright © 2020 thepermanentejournal.org.

All Rights Reserved.