Understanding Faculty and Trainee Needs Related to Scholarly Activity in a Large, Nonuniversity Graduate Medical Education Program

Tweet

Davida Becker, PhD, MS; Hanna Garth, PhD, MPH; Rachel Hollander;
Felice Klein, RN, MN; Marc Klau, MD, MBA

Perm J 2017;21:16-034 [Full Citation]

https://doi.org/10.7812/TPP/16-034

E-pub: 01/06/2017

ABSTRACT

Context: Graduate medical education (GME) programs must develop curriculum to ensure scholarly activity among trainees and faculty to meet accreditation requirements and to support evidence-based medicine.
Objective: Test whether research-related needs and interests varied across four groups: primary care trainees, specialty trainees, primary care faculty, and specialty faculty.
Design: We surveyed a random sample of trainees and faculty in Kaiser Permanente Southern California’s GME programs. We investigated group differences in outcomes using Fisher exact and Kruskal-Wallis tests.
Main Outcome Measures: Research experiences, skills, barriers, motivators, and interests in specific research skills development.
Results: Participants included 47 trainees and 26 faculty (response rate = 30%). Among primary care faculty, 12 (71%) reported little or no research experience vs 1 (11%) for specialty faculty, 14 (41%) for primary care trainees, and 1 (8%) for specialty trainees (p < 0.001). Submission of research to the institutional review board, an abstract to a conference, or a manuscript for publication in the previous year varied across groups (p = 0.001, p = 0.003, and p < 0.001, respectively). Overall self-reported research skills also differed across groups (p < 0.001). Primary care faculty reported the lowest skill level. Research barriers that differed across groups included other work roles taking priority; desire for work-life balance; and lack of managerial support, research equipment, administrative support, and funding.
Conclusion: Faculty and trainees in primary care and specialties have differing research-related needs that GME programs should consider when designing curricula to support scholarly activity. Developing research skills of primary care faculty is a priority to support trainees’ scholarly activity.

INTRODUCTION

Participation in scholarly activity during residency training benefits trainees by promoting the practice of evidence-based medicine and quality patient care, providing skills for lifelong learning, and supporting critical thinking skills.¹ Additionally, participation in research may be necessary for residents interested in fellowship placements.¹ The Accreditation Council of Graduate Medical Education (ACGME) mandates participation in scholarly activity for residents and faculty in all specialties, and some specialty review committees have specified additional requirements.² Fulfilling the scholarly activity requirement means graduate medical education (GME) programs must develop curriculum and structures that support research, address research barriers, and foster a culture of inquiry.^1,3 The most effective and efficient programs address learners’ needs and preferences, and account for the current level of research experience of targeted groups.^4-6

For large GME institutions with training programs in diverse specialties, developing programs to support scholarly activity may present challenges if needs and preferences vary across groups. Barriers to research may be different or more pronounced in primary care programs, where levels of research experience and skills may be lower than in specialties.^3,7-9 Another complexity is that faculty may have different training needs than trainees do because their role involves both conducting research and mentoring trainees’ scholarly activities.

Previous studies have documented trainees’,^8,10-12 program directors’,^7,13 and practicing physicians’^9,14 perspectives on research and the scholarly activity requirements. However, most studies were limited to a single specialty and population, such as residents or program directors. Few investigators have compared faculty and trainee perspectives across various specialties in a single study.

Kaiser Permanente Southern California (KPSC) is a large integrated health care system that provides care to more than 4 million individuals across Southern California at 14 Medical Centers and 221 medical offices. At 6 of these Medical Centers, KPSC sponsors 32 independent ACGME-accredited residency and fellowship programs, most (n = 19) of which are based at the Los Angeles Medical Center (LAMC). The other 13 programs are located at Medical Centers across the Region, including Fontana, Orange County, Riverside, Woodland Hills, and San Diego, CA. LAMC is where most specialty training takes places and as such has the greatest number of physicians engaged in research. Of all the KPSC-sponsored programs, 11 are primary care programs, including 6 Family Medicine, 2 Internal Medicine, 1 Pediatrics, and 2 Geriatrics programs. Each year, KPSC graduates around 114 trainees, approximately 60% from primary care programs.

In 2014, KPSC’s GME administration started a program to build research capacity in the GME programs. To inform the development of the program, a survey of faculty and trainees was conducted to measure research-related experiences, skills, barriers, motivators, and interest in skills development. We used data from this survey to test for differences in research-related needs and interests across four groups: primary care trainees, primary care faculty, specialty trainees, and specialty faculty. We hypothesized that research-related experiences, skills, and interests would be different for faculty and trainees affiliated with primary care programs compared with those in specialties.

METHODS

We surveyed a random sample of faculty and trainees between January 2015 and July 2015. Eligibility criteria included primary affiliation with a KPSC GME program. Faculty and trainees were sampled separately. The sampling frames were administrative lists of all current residents and faculty. The GME office administered the survey as a baseline assessment to evaluate the research capacity building program launched in 2014; a follow-up survey is planned in 5 years. We estimated the sample size to detect a 15% increase in research-related skills from baseline to follow-up with 80% power and adjusted it for an anticipated 50% nonresponse rate. The final sample size was 160 trainees and 90 faculty. The KPSC institutional review board approved the study protocol.

Participants were Emailed a link to an online survey (SurveyMonkey, Palo Alto, CA) from a Kaiser Permanente Email address and were sent 3 reminder Emails over 2.5 months. We used a SurveyMonkey feature that allowed for tracking whether or not participants had responded but which stored all survey data anonymously. Because response to the online survey was lower than expected, we distributed printed copies of the survey to all nonresponders with instructions to return their survey in a sealed envelope using our interoffice mail system. Participants who completed the survey were offered the option to be entered into a raffle to win 1 of 2 backpacks (approximate value $50 each).

The questionnaire was adapted from the Research Capacity and Culture Tool, a validated instrument with high reliability that has been used previously to evaluate research capacity building initiatives.^15,16 Domains measured included research experience, research activities in the previous year, research skills, barriers and motivators to research, and interests in research skills development. We also measured program affiliation, role (trainee or faculty), age, sex, years of clinical experience, and primary training site. Participants who selected Family Medicine, Internal Medicine, Pediatrics, or Geriatrics as their program affiliation were categorized as primary care.¹⁷ Residents and fellows were grouped as trainees. All participants were categorized into four mutually exclusive groups: primary care trainees, specialty trainees, primary care faculty, and specialty faculty.

We performed statistical analysis using Stata Version 14 (College Station, TX).¹⁸ First, we compared characteristics of faculty and trainees in the sample with those who were included in the pool of eligible participants to assess for potential biases. Next, we estimated proportions for categorical variables, and medians and interquartile ranges for ordinal measures. We created a summary measure of overall self-reported research skills by averaging responses to individual questions on specific research skills (Cronbach a = 0.96). We tested for differences in research-related skills, experience, barriers, motivators, and interests across the 4 groups using the Fisher exact test and Kruskal-Wallis test. P values less than 0.05 were considered statistically significant.

RESULTS

Three individuals sent surveys were ineligible because of termination of a GME affiliation or medical leave. Eighty-five individuals responded to the survey; 50 participants responded to the Email survey, and 35 completed a paper copy. We excluded 10 participants who initiated a survey but left three-fourths of the questions blank and 2 participants who did not provide data on their program or role (trainee or faculty). The analytic sample comprised 73 participants, including 47 trainees and 26 faculty. The response rate for both the trainee and faculty samples was 30%, 47 of 159 trainees and 26 of 88 faculty. There were no significant differences between faculty and trainees in the sample and those in the pool of eligible participants by program affiliation (primary care vs specialty), site (LAMC vs other sites), or sex (men vs women).

Participants’ program affiliations included the following: Family Medicine (n = 34), Internal Medicine (n = 7), Pediatrics (n = 5), Geriatrics (n = 5), Urology (n = 6), Orthopedics/Sports Medicine (n = 6), Emergency Medicine (n = 2), Obstetrics/Gynecology (n = 2), Gastroenterology (n = 1), Nephrology (n = 1), Neurology (n = 1), Hospice/Palliative Medicine (n = 1), and Diagnostic Radiology (n = 1). Program information was missing from 1 participant.^a Thirty-five (74%) trainees and 17 (65%) faculty were affiliated with a primary care program (Table 1). Among trainees, 24 (51%) were based at LAMC compared with 10 (38%) of faculty. Twenty-one (45%) trainees were men compared with 19 (73%) faculty.

Research Experience and Skills

Levels of research experience varied by role and program (Table 2). Among primary care faculty, 12 (71%) reported “little or no” research experience compared with 1 (11%) specialty faculty, 14 (41%) primary care trainees, and 1 (8%) specialty trainee (p < 0.001). Six (35%) primary care faculty and 6 (67%) specialty faculty reported mentoring a trainee’s research in the previous year. Among specialty faculty who had mentored trainee research in the previous year, 4 (67%) reported “substantial” research experience and 2 (33%) reported “some” research experience. Among primary care faculty who had mentored trainee research in the previous year, 1 (17%) reported “substantial” research experience, 4 (67%) reported “some” research experience, and 1 (17%) reported “little or no” research experience.

When asked to report research activities undertaken in the previous year, the percentage who initiated a research study or a quality-improvement (QI) project and who participated in someone else’s research or QI project did not differ by role or program. However, there were differences in the percentage who submitted research to the institutional review board (p = 0.001), an abstract to a conference (p = 0.003), and a manuscript to a peer-reviewed journal (p < 0.001). Primary care faculty and trainees had the lowest percentages for completing these activities.

When survey participants were asked to evaluate their individual research skills on a scale from 1 to 10 (with 10 being the highest skill), ratings differed across groups for all skills except data collection (Table 3). Median scores for overall self-reported research skills also varied across groups: 3.6 among primary care faculty, 7.4 among specialty faculty, 5.7 among primary care trainees, and 6.7 among specialty trainees (p < 0.001). Median scores for overall self-reported research skills also differed across research experience groups; among participants with “little or no” research experience, the median score was 3.3 compared with 6.0 for those who had “some” research experience, and 7.2 for those who had “substantial” research experience (p < 0.001).

Research Barriers and Motivators

More than one-fifth of faculty and trainees identified the following factors among the top 3 barriers to research (Table 4): no dedicated time, 46 (63%); other work roles take priority, 33 (45%); desire for work-life balance, 18 (25%); not interested, 17 (23%); and lack of skills, 17 (23%). Of these, other work roles taking priority (p = 0.012), and a desire for work-life balance (p = 0.026) differed significantly across groups. Other barriers to research that differed across groups included lack of managerial support (p = 0.002), limited access to research equipment (p = 0.032), lack of administrative support (p = 0.003), and lack of funding (p = 0.008).

Factors identified by more than one-fifth of faculty and trainees as 1 of the top 3 motivators for research included the following: improving clinical practice, 29 (40%); a problem that needs changing, 27 (37%); developing skills, 23 (32%); improving critical thinking skills, 19 (26%); career advancement, 18 (25%); job satisfaction, 18 (25%); and keeping the brain stimulated, 17 (23%; Figure 1). The only motivator that differed across groups was developing skills (not shown): primary care trainees, 12 (34%); specialty trainees, 8 (67%); primary care faculty, 3 (18%); and specialty faculty, 0 (0%); (p = 0.005).

Research Skills Development Interests

When asked to rate their interests in developing specific research skills, more than two-thirds of faculty and trainees reported being “somewhat” or “very” interested in developing skills in the following areas: applying research findings to clinical practice, 59 (81%); finding relevant literature, 58 (79%); generating research ideas, 56 (77%); conducting a study using electronic medical record data, 55 (75%); educating and/or communicating research findings to patients, 54 (74%); analyzing and interpreting results, 53 (73%); critically reviewing literature, 51 (71%); publishing on a QI project, 50 (68%); and writing a case report or case series, 48 (67%; Figure 2).

Level of interest in developing specific research skills did not differ across groups except for using quantitative research methods (not shown): primary care trainees, 17 (49%); specialty trainees, 11 (92%); primary care faculty, 11 (65%); and specialty faculty, 7 (78%); (p = 0.039). Among primary care faculty, 8 (47%) reported being “somewhat” or “very” interested in developing research mentorship skills; this was 5 (56%) among specialty faculty. Faculty interest in developing research mentorship skills varied by level of research experience. Among faculty with “little or no” research experience, 3 (23%) expressed interest in developing research mentorship skills compared with 5 (71%) faculty with “some” research experience and 5 (83%) faculty with “substantial” research experience (p = 0.033). Among faculty who had mentored trainee research in the previous year, 10 (83%) expressed interest in developing their research mentorship skills, and interest was high, even among faculty reporting “substantial” (4 [80%]), or “some” (5 [83%]) research experience themselves.

DISCUSSION

To meet ACGME requirements for scholarly activity, GME institutions must develop appropriate curricula and infrastructure to support scholarly work. Large institutions with training programs in multiple specialties face challenges when research-related training needs vary across groups, making it necessary to develop targeted programming; however, few previous studies have examined differences in research-related needs across specialties or roles (faculty vs trainees). To better understand these, we tested for differences in research-related experiences, skills, and interests among four groups: primary care trainees, specialty trainees, primary care faculty, and specialty faculty. We found significant differences in the amount and types of experience, skill levels, and several perceived barriers to conducting research. However, there were few differences across groups in factors motivating research involvement or specific research skills that participants were interested in developing.

Overall, primary care groups reported having less research experience and fewer research skills than did specialty groups, with the lowest levels among primary care faculty. This finding is consistent with those of previous studies,^3,14,19 highlighting the challenge primary care programs face to meet ACGME scholarly activity requirements and to provide adequate research mentorship.^1,3 For primary care-rich institutions like ours, these results suggest that an efficient GME research program focuses on building research capacity in primary care programs. A long-term goal for programs similar to ours could be to develop primary care faculty “research champions.” Others have recognized the essential role that faculty and program directors play as role models and mentors for research.^3,20,21 The current shortage of primary care faculty with research experience and skills limits faculty members’ own research productivity and results in faculty mentors for trainee research who lack relevant knowledge and skills. This situation may contribute to unsuccessful or incomplete projects and dissatisfaction with the research experience for both trainees and faculty.

A noteworthy finding is that faculty members’ interest in developing research mentorship skills was associated with their own research experience. Faculty with at least some research experience had higher interest in developing research mentorship skills than did those with little or no experience. A fruitful path forward may be to expose primary care faculty to research experience, which may result in increasing their own research skills and increasing their interest in mentoring trainee research.

The top reported barriers to conducting research reflected individual factors such as lack of time, skills, and interest, as well as institutional factors, including lack of managerial support and equipment. Previous studies have reported similar barriers,^1,4,11 supporting a multilevel approach to building research capacity addressing both individual and organizational factors.⁵ Our results indicated differences across groups for several barriers, suggesting that effective programs should incorporate strategies to mitigate barriers for particular groups. Other than the top barrier of no dedicated time, specialty faculty did not select any of the next four most frequently cited barriers, all of which represent individual factors. Institutional barriers appear to be more salient for specialty faculty.

The most frequently cited motivators for research were improving clinical practice and solving an existing problem. Other motivators included developing skills, improving critical thinking, and career advancement. These findings are consistent with those of previous studies.^11,14 Research programs might motivate research participation by framing research as relevant to clinical practice and acknowledging intrinsic rewards such as personal and professional development.

When participants were asked to rate their level of interest in specific research skills, many of the skills rated of highest interest related to practicing evidence-based medicine. This supports incorporating evidence-based medicine topics in a research fundamentals curriculum. We identified few differences in the skills of interest across groups, suggesting the feasibility of using an institutionwide core curriculum for research. Additionally, a high percentage of faculty who mentored trainees on research were interested in developing their research mentorship skills, and this included faculty who had research experience themselves. This finding highlights the fact that skills needed to mentor research are different from those needed to conduct research. Faculty development on research mentorship skills would be valuable.

Our study has several limitations. Because the study was designed for program evaluation and not to test for group differences in outcomes, we had small sample sizes in each comparison group. Our results should be considered exploratory and must be validated in a larger study designed specifically to investigate group differences. Additionally, the response rate was low at 30%, although comparable with other surveys of physicians.²² Participants may have also underreported or overreported their research experience and skills. Finally, we grouped participants into broad categories of primary care or specialty and trainee or faculty, and differences may exist in each of these categories.

CONCLUSION

Institutions of GME should address differences in research-related needs of targeted groups when designing curricula to support scholarly activity. Primary care faculty, primary care trainees, specialty faculty, and specialty trainees were found to differ in amount and types of research experience, research skill levels, and several perceived barriers to research. However, there were few differences across groups in factors motivating research involvement or in the specific research skills that participants were interested in developing. Primary care faculty consistently reported the lowest levels of research skills and experience. Targeted efforts to develop research capacity among primary care faculty are warranted.

a Participant specified his/her role at a Medical Center with only primary care programs and could therefore be included in the analysis.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Acknowledgments

Kathleen Louden, ELS, of Louden Health Communications provided editorial assistance.

How to Cite this Article

Becker D, Garth H, Hollander R, Klein F, Klau M. Understanding faculty and trainee needs related to scholarly activity in a large, nonuniversity graduate medical education program. Perm J 2017;21:16-034. DOI: https://doi.org/10.7812/TPP/16-034.

References

1.    Rothberg MB. Overcoming the obstacles to research during residency: What does it take? JAMA 2012 Dec 5;308(21):2191-2. DOI: https://doi.org/10.1001/jama.2012.14587.
    2.    Philibert I, Lieh-Lai M, Miller R, Potts JR 3rd, Brigham T, Nasca TJ. Scholarly activity in the next accreditation system: Moving from structure and process to outcomes. J Grad Med Educ 2013 Dec;5(4):714-7. DOI: https://doi.org/10.4300/jgme-05-04-43.
    3.    Seehusen DA, Weaver SP. Resident research in family medicine: Where are we now? Fam Med 2009 Oct;41(9):663-8.
    4.    Hebert RS, Levine RB, Smith CG, Wright SM. A systematic review of resident research curricula. Acad Med 2003 Jan;78(1):61-8. DOI: https://doi.org/10.1097/00001888-200301000-00012.
    5.    Cooke J. A framework to evaluate research capacity building in health care. BMC Fam Pract 2005 Oct 27;6:44. DOI: https://doi.org/10.1186/1471-2296-6-44.
    6.    Farmer E, Weston K. A conceptual model for capacity building in Australian primary health care research. Aust Fam Physician 2002 Dec;31(12):1139-42.
    7.    DeHaven MJ, Wilson GR, Murphree DD, Grundig JP. Family practice residency program directors’ views on research. Fam Med 1997 Jan;29(1):33-7.
    8.    Temte JL, Hunter PH, Beasley JW. Factors associated with research interest and activity during family practice residency. Fam Med 1994 Feb;26(2):93-7.
    9.    Leahy N, Sheps J, Tracy CS, Nie JX, Moineddin R, Upshur RE. Family physicians’ attitudes toward education in research skills during residency: Findings from a national mailed survey. Can Fam Physician 2008 Mar;54(3):413-4.
    10.    Ledford CJ, Seehusen DA, Villagran MM, Cafferty LA, Childress MA. Resident scholarship expectations and experiences: Sources of uncertainty as barriers to success. J Grad Med Educ 2013 Dec;5(4):564-9. DOI: https://doi.org/10.4300/jgme-d-12-00280.1.
    11.    Rivera JA, Levine RB, Wright SM. Completing a scholarly project during residency training. Perspectives of residents who have been successful. J Gen Intern Med 2005 Apr;20(4):366-9. DOI: https://doi.org/10.1111/j.1525-1497.2005.04157.x.
    12.    Neacy K, Stern SA, Kim HM, Dronen SC. Resident perception of academic skills training and impact on academic career choice. Acad Emerg Med 2000 Dec;7(12):1408-15. DOI: https://doi.org/10.1111/j.1553-2712.2000.tb00499.x.
    13.    Levine RB, Hebert RS, Wright SM. Resident research and scholarly activity in internal medicine residency training programs. J Gen Intern Med 2005 Feb;20(2):155-9. DOI: https://doi.org/10.1111/j.1525-1497.2005.40270.x.
    14.    Bakken S, Lantigua RA, Busacca LV, Bigger JT. Barriers, enablers, and incentives for research participation: A report from the Ambulatory Care Research Network (ACRN). J Am Board Fam Med 2009 Jul-Aug;22(4):436-45. DOI: https://doi.org/10.3122/jabfm.2009.04.090017.
    15.    Holden L, Pager S, Golenko X, Ware RS. Validation of the research capacity and culture (RCC) tool: Measuring RCC at individual, team and organisation levels. Aust J Prim Health 2012;18(1):62-7. DOI: https://doi.org/10.1071/py10081.
    16.    Howard AJ, Ferguson M, Wilkinson P, Campbell KL. Involvement in research activities and factors influencing research capacity among dietitians. J Hum Nutr Diet 2013 Jul;26 Suppl 1:180-7. DOI: https://doi.org/10.1111/jhn.12053.
    17.    Primary care workforce facts and stats no. 1. The number of practicing primary care physicians in the United States. AHRQ pub no. 12-P001-2-EF [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; 2011 Oct [cited 2015 Nov 12]. Available from: www.ahrq.gov/sites/default/files/publications/files/pcwork1.pdf.
    18.    StataCorp. Stata statistical software: Release 14. College Station, TX: StataCorp LP; 2015.
    19.    Jones A, Burgess TA, Farmer EA, et al. Building research capacity. An exploratory model of GPs’ training needs and barriers to research involvement. Aust Fam Physician 2003 Nov;32(11):957-60.
    20.    DeHaven MJ, Wilson GR, O’Connor-Kettlestrings P. Creating a research culture: What we can learn from residencies that are successful in research. Fam Med 1998 Jul-Aug;30(7):501-7.
    21.    Crawford P, Seehusen D. Scholarly activity in family medicine residency programs:
A national survey. Fam Med 2011 May;43(5):311-7.
    22.    Grava-Gubins I, Scott S. Effects of various methodologic strategies: Survey response rates among Canadian physicians and physicians-in-training. Can Fam Physician 2008 Oct;54(10):1424-30.