Health Care Systems Support to Enhance Patient-Centered Care: Lessons from a Primary Care-Based Chronic Pain Management Initiative

Health Care Systems Support to Enhance Patient-Centered Care: Lessons from a Primary Care-Based Chronic Pain Management Initiative

 

Charles R Elder, MD, MPH, FACP; Lynn L DeBar, PhD, MPH; Cheryl Ritenbaugh, PhD, MPH,
Maureen H Rumptz, PhD; Charlotte Patterson, MBCHB, PhD, MBAcC, MRCGP; Allison Bonifay, MA, LPC;
Penney Cowan; Lindsay Lancaster, RN, PhD, CNS; Richard A Deyo, MD, MPH

Perm J 2017;21:16-101 [Full Citation]

https://doi.org/10.7812/TPP/16-101
E-pub: 04/03/2017

ABSTRACT

Background: Supporting day-to-day self-care activities has emerged as a best practice when caring for patients with chronic pain, yet providing this support may introduce challenges for both patients and primary care physicians. It is essential to develop tools that help patients identify the issues and outcomes that are most important to them and to communicate this information to primary care physicians at the point of care.
Objective: We describe our process to engage patients, primary care physicians, and other stakeholders in the context of a pilot randomized controlled trial of a patient-centered assessment process implemented in an everyday practice setting. We identify lessons on how to engage stakeholders and improve patient-centered care for those with chronic conditions within the primary care setting.
Methods: A qualitative analysis of project minutes, interviews, and focus groups was conducted to evaluate stakeholder experiences. Stakeholders included patients, caregivers, clinicians, medical office support staff, health plan administrators, an information technology consultant, and a patient advocate.
Results: Our stakeholders included many patients with no prior experience with research. This approach enriched the applicability of feedback but necessitated extra time for stakeholder training and meeting preparation. Types of stakeholders varied over the course of the project, and more involvement of medical assistants and Information Technology staff was required than originally anticipated.
Conclusion: Meaningful engagement of patient and physician stakeholders must be solicited in a well-coordinated manner with broad health care system supports in place to ensure full execution of patient-centered processes.

Introduction

The management of chronic health conditions continues to dominate primary care concerns in the US. More than 50% of American adults have at least one chronic condition, and 25% of adults have two or more chronic health conditions.1 Chronic musculoskeletal pain has been identified as a substantial public health concern, with spinal disorders representing the fourth-most-common primary diagnosis leading to office visits.2 Effective chronic disease management necessitates patients’ active day-to-day involvement in their health.3 The increasing need for health care systems to address chronic conditions highlights the importance of patients’ engagement with their primary care physicians (PCPs).4 The need for such engagement is most evident in chronic pain management. Although supporting patients in their day-to-day self-care practices has emerged as a best practice for patients with nonmalignant chronic pain,5,6 achieving an ideal level of support can be fraught with challenges for both patients and PCPs. More research is needed to identify procedures and practices that more effectively empower patients and clinicians as they work together to optimize patients’ self-care efforts. Patient and clinician stakeholders must be involved in research projects that address the provision of chronic pain management in primary care to develop an evidence base that is practical and relevant to health care system end users.

Involvement of patient and clinician stakeholders in research has increased in recent years.7,8 However, most efforts have placed little emphasis on the broader context of the health care setting, and experience and precedent are limited regarding ways to engage frontline stakeholders in comparative clinical effectiveness research.

We conducted preliminary research9,10 that generated qualitative data highlighting complementary themes cited by patients with chronic pain, their PCPs, and Health Plan administrators. Although patients frequently recognized that their PCPs had limited tools with which to ameliorate their pain, they emphasized the importance of “feeling listened to and supported” by their PCP in managing their condition. PCPs acknowledged the emotional toll of caring for patients with pain but remained enthusiastic about their participation in decisions affecting their care. Health Plan administrators were most concerned about identifying optimal care for patients with complex needs. Collectively, this research showed that neither patients nor their PCPs were satisfied with their level of care despite the resources invested. Their concerns suggest the need for health care structural supports to identify and to act on patient-prioritized needs to improve care experiences and outcomes.

In this article, we describe the process we used to engage patients, PCPs, and other stakeholders to implement a pilot randomized controlled trial of a patient-centered assessment process for chronic pain management in an everyday practice setting. Results of the clinical trial will be reported elsewhere. In this article, we identify lessons learned from our experience in engaging patients and clinicians and supporting stakeholders as partners in research to improve patient-centered care within the primary care setting.

METHODS

Overall Study Design

This study was sponsored by the Patient-Centered Outcomes Research Institute. Our study sought to have researchers fully partner with relevant stakeholders, including patients with complex chronic pain and their PCPs and staff who support physician-patient relationships. These included medical assistants (MAs), nursing staff, administrators, and Information Technology (IT) personnel who programmed features into the electronic medical record (EMR).

The project had two goals: to examine the effect of using a patient-centered instrument on the satisfaction of patients and PCPs within the primary care encounter and to examine whether use of this tool and process affected longer-term pain-related outcomes. We viewed engagement of patients, physicians, and supporting stakeholders as essential to achieving these goals. We adapted the Measure Yourself Medical Outcome Profile (MYMOP2),11,12 a validated patient-generated assessment instrument, for use within the primary care environment for patients with complex chronic pain conditions who have been receiving long-term opioid therapy. The adapted tool measured the outcomes patients considered most important and framed the issues using their own words. The adapted tool and pilot trial results are described in a separate manuscript. The focus here is on the approach taken and lessons learned in the stakeholder engagement process.

Study Setting

Kaiser Permanente Northwest serves about 540,000 members in northwest Oregon and southwest Washington. Members’ demographic characteristics mirror the surrounding population, which is 76% Caucasian, 9% Latino, 7% Asian, and 6% African American,13 and their health status scores (on Short Form-36 Health Survey subscales) are similar to US statistics.14 The Kaiser Permanente Northwest integrated care system includes complete EMRs for each member. Kaiser Permanente Northwest has 18 primary care clinics and more than 300 PCPs; this study was conducted in two of these clinics.

Participants

We included patient stakeholders, caregivers, and a nationally recognized patient advocate. We also included PCPs, MAs, nurses, administrators from the clinic at which the clinical trial was initiated, and an IT representative. Our 6 patient stakeholders ranged from age 45 to 85 years, were balanced by sex, and had complex chronic pain. Patients were deemed to have chronic complex pain where there was EMR evidence of receipt of long-term (> 3 months) opioid medication and an EMR diagnosis during the last year of either a widespread pain condition such as fibromyalgia or a comorbid mood diagnosis in addition to a musculoskeletal pain diagnosis. Those with a cancer-related pain diagnosis were excluded.

Five of the patient stakeholders were Caucasian, and one was African American. We selected patient stakeholders who did not have previous personal, advocacy, or professional experience with clinical research. This allowed us to prioritize more modal patient treatment experiences and ascertain how successful such patients could be while engaged in research. We provided patient stakeholders with research training during the course of the study, including information about the study’s original design, measurement development, data interpretation, dissemination, and general research processes. Our institutional review board classified patient stakeholders as “research team members” rather than “human subjects.” Further, having a patient advocate on our research team allowed us to consider care processes across a broader set of health care settings and patient experiences. Two people who had spouses with chronic pain were involved in the study to provide a caregiver perspective. Other stakeholders included PCPs, clinic administrators, medical assistants, and nurses.

Approach and Analysis

Data collected during project meetings with our clinician and patient stakeholders constituted an important source of information. Research staff also conducted individual interviews with four PCPs and focus groups with three MAs, two patient caregivers, and two behavioral health interviewers/interventionists from the research team, respectively, using preprepared interview guides. With the exception of the patient caregivers, all those participating in interviews and focus groups participated in the pilot trial. The interview guides were tailored to particular subjects and focused on issues surrounding the implementation of the tool; perceived barriers and benefits; possible modifications in the tool; or procedures that would have enhanced its uptake, usability, and impact. Comprehensive minutes of all meetings, interviews, and focus groups provided the material for subsequent theme analyses.

The International Association for Public Participation (IAP2) spectrum approach for public participation15 provided the organizing framework with which to identify levels of engagement of constituent stakeholder groups (see Sidebar: Stakeholder Engagement [International Association for Public Participation’s IAP2 criteria]15). Widely used throughout the health care system, the IAP2 approach was familiar to many of our stakeholders.

To avoid premature imposition of a prescribed conceptual framework for stakeholder engagement, four members of the research team conducted preliminary reviews of all meeting, interview, and focus group data. Each member independently summarized emergent themes from the data and identified patterns. Team members discussed these findings and agreed-on key themes and patterns through an iterative process.16,17 The conceptual framework of Oliver and colleagues,18 which consists of three related dimensions (involvement drivers, processes, and impact), provided a model that could be used to describe stakeholder engagement in clinical research and to organize themes and patterns within the data.

Health Care Systems Support to Enhance Patient-Centered Care: Lessons from a Primary Care-Based Chronic Pain Management Initiative

RESULTS

Framework for Stakeholder Involvement

Figure 1 presents the framework for stakeholder involvement. The process of engaging patients with chronic complex pain ensured our study design, intervention, data collection, data analyses, and results dissemination were relevant, sensitive, and of major importance to patients. A national-level patient advocate for chronic pain care offered a patient’s perspective and invaluable insight on the generalizability of our process and findings beyond our own health care system. Patient caregivers offered complementary perspectives. Clinical stakeholders included PCPs (the primary point of contact for patients and their treatment), MAs, and nurses. Clinic managers provided valuable direct evidence on clinical workflow. At the health care system level, operational leaders identified the original need for this intervention and informed key decisions about the adoption of practice improvements. Finally, our IT representative created ways to embed processes into the EMR to efficiently integrate patient information from the study into the clinical workflow.

Patient stakeholders received support, tailored material, and $25 gift cards as compensation to attend each meeting. Several research staff met with these patients during separate bimonthly meetings. The research team observed, and patients reported, that they felt most comfortable when involved in smaller meetings held at their primary care clinic. The in-person meetings allowed us to use a variety of visual aids to discuss essential study elements and processes, which ensured equitable access to study information. These participants received important foundational training in study design, institutional review board processes, and funder expectations and relevant information about chronic pain treatment to prepare them as partners in the process. Although a subset of patient stakeholders also joined the larger research meetings at multiple junctures, this smaller group allowed time for co-learning; researchers provided information about the study, original design, and general research processes, and patient stakeholders provided information about their lived experience. Patient stakeholders offered each other informal support as they shared experiences, and this support helped to create the trust and confidence necessary to challenge traditional care processes.

From a practical standpoint, clinical stakeholders had to be engaged during their busy workdays. Several members of the research team joined morning department meetings and team huddles, but many PCPs and their support staff missed meetings because of their busy clinical schedules. Research team members’ brief communications at each interaction emphasized the shared goal of reducing clinician burden and better serving patients by enhancing patient-centered assessments at the point of care.

An IT consultant from the Health Plan advised the research team on ways to most effectively embed the instrument into the EMR. Because the revised instrument as deployed in the clinical trial was somewhat unconventional (eg, including open-text fields), the IT partner’s role was especially valuable. The research team worked with the IT consultant to frame the rationale for the tool’s clinical use, identify an efficient way to attach the instrument to the clinical encounter within the EMR, and guide the execution process within the Health Plan’s IT department.

Comments Regarding the Framework

Stakeholder involvement affected the entire research process and resulted in substantial changes in the MYMOP-based primary assessment tool, including simplification of response categories for some questions and use of tailored, graduated levels to express a patient’s chosen goal. Stakeholder feedback also allowed the research team to build a feasible infrastructure to integrate the necessary intervention components into the clinical workflow and EMR. Working with stakeholder partners necessitated considerable time and flexibility. Despite these challenges, this approach resulted in a better study design, a more meaningful tool, and novel avenues for dissemination of study findings. Similarly, patient stakeholders reported a sense of ownership, achievement, and confidence as they advocated for their needs. PCPs and staff gained experience with and appreciation for the practical challenges of implementing practice improvement interventions in a scientific manner. Finally, the research investigators gained an understanding of patients’ experiences with chronic pain and the things the patients most needed from the health care delivery system. Investigators also gained an appreciation for the challenges clinicians face when changing the ways in which they provide care. Overall, the reciprocal transfer of knowledge, skills, capacity, and power were evident to study stakeholders.

Stakeholder Interactions During the Course of Study

Figure 2 provides a temporal illustration of the level at which we engaged each of our stakeholder groups across time and the ways in which feedback influenced future interactions. Frequent interaction across many of the groups was the norm, and the figure highlights the interactions most consequential to the evolving approach and research product. Level of involvement reflects the International Association for Public Participation’s IAP2 criteria15 as summarized in the Sidebar: Stakeholder Engagement (International Association for Public Participation’s IAP2 criteria).15

During prestudy planning, collaboration with Health Plan administrative leaders and our PCP partners was vital. The concerns expressed by regional leaders about gaps in the care of patients with complex chronic pain provided project direction, and their permission was needed to initiate the study (leaders needed to prioritize the efforts of the frontline clinical staff). Leaders’ concerns, PCP partners’ clinical experiences and suggestions for feasible study approaches, and patient reports of their needs set the stage for the project.

Once the study was underway, group members interacted more often at patient stakeholder meetings and formed relationships that stimulated rich discussions about patient care and experiences with past and current physicians. The patient stakeholder group influenced the study design by offering feedback on the main outcome instrument, from wording changes to recharacterization of response categories from a Likert scale format to a goal attainment scaling approach.19 The research team notified PCP partners and clinical support staff of substantive changes in approach so they were aware of modifications. This is an example of how unanticipated stakeholder-driven processes changed the nature of the interactions and study communications during subsequent study phases.

The project’s randomized controlled pilot phase necessitated more involvement of clinical support staff, including MAs, nurses, and clinic managers, to reflect on their respective roles in managing clinical workflow in the primary care setting. The IT consultant role was larger than initially anticipated because it involved building of the EMR-based structures. The terminology and frames of reference were different for clinical research and IT; consequently, time was devoted to develop a common understanding about specific needs and viable pathways.

Health Care Systems Support to Enhance Patient-Centered Care: Lessons from a Primary Care-Based Chronic Pain Management Initiative

Health Care Systems Support to Enhance Patient-Centered Care: Lessons from a Primary Care-Based Chronic Pain Management Initiative

DISCUSSION

We learned a number of lessons in the conduct of this study. First, focusing on a tool to ease and to bolster PCP and patient interactions necessitated broad infrastructure and staffing support during primary care clinic visits. As such, we could not rely solely on PCPs or patients to drive the process. PCPs are under intense pressure to address many competing demands, and it is difficult to capture their time and attention to ensure they understand the expected process at the point of care. IT support was crucial to devise a way to embed the assessment summary within the EMR. Such tools are increasingly embedded in EMRs to guide the flow of clinical work. Because of their roles as extenders for PCPs, MAs were also essential study stakeholders. Early in the study, we failed to appreciate the importance of MAs in facilitating the clinical workflow process for PCPs, and we should have enlisted their involvement sooner. Patients often look to MAs as the cornerstone of their medical care team, and they often have telephone contact with them between in-person encounters; this relationship has become more important because fewer nurses are included on today’s primary care teams. Ensuring that MAs were familiar with the content and rationale for the study-designed tool and encouraging them to provide consistent patient-related messaging may have increased the study’s interaction quality. However, it may be challenging to build in a systematic role for MAs to deliver and/or to reinforce this type of process because of their practice restrictions and role variations across primary care clinics. Potential obstacles include MAs’ limited medical training, union regulations regarding scope and timing of work, and the fact that they are employed by a different entity than the PCPs in this health care system. Finally, high turnover among key clinical and operational leaders led to less centralized high-level support for the ongoing project. Because leadership tends to change often, embedded clinical trials and patient-centered work should be designed to plan for the ongoing need to refresh relationships with operational leadership and actively maintain their support. Collectively, these experiences suggest that although patient stakeholder input is essential, patients’ recommendations can be enacted only if clinical and health care system factors can accommodate their demands.

We also learned that including many types of stakeholders and modes of identifying and recruiting patient-, caregiver-, and clinician-stakeholders presented both benefits and liabilities. The inclusion of a variety of patients and clinical stakeholders on the research team improved the entire team’s awareness of stakeholder-valued issues. We chose patients with no prior research experience so we could get a truly patient-centric view of the process. Although this approach enriched the applicability of feedback, it necessitated more time spent in stakeholder training and meeting preparation. Although community-based participatory research often empowers stakeholders with decision-making authority at all points in the process, this approach is unlikely to be practical or effective in medical settings.

Finally, before beginning the study, all of our stakeholders identified a more systematic process to manage complex chronic pain conditions as a priority. As a result, this study’s focus was truly patient-and stakeholder-centered. Developing this process, however, was complicated from the vantage point of all stakeholders, and it was difficult to comprehensively address all concerns in the context of a relatively small pilot study. Furthermore, growing organizational concerns surrounding opioid prescribing were being addressed while this study was in progress, so many simultaneous efforts were being implemented within the health care system. This phenomenon further complicated the research team’s efforts to secure resources and ensure study activities could be clearly demarcated by affected clinical staff from these broader initiatives.

Although plentiful literature on community-based participatory research exists, scant literature guides or describes the process of engaging patient stakeholders as partners in biomedical comparative effectiveness research. Forsythe and colleagues20 surveyed principal investigators of the 50 initial pilot projects funded by the Patient-Centered Outcomes Research Institute, of which this study was a part, describing aggregate data for the types of stakeholders engaged and their levels of engagement. Forsythe and colleagues21 likewise surveyed primary care clinicians and patients to garner information about attitudes toward participation as stakeholders in comparative effectiveness research, a majority of whom indicated interest in engagement as research partners. Other efforts include studies describing patient engagement in cholesterol screening for adolescents22 and in cardiovascular disease clinical comparative effectiveness research.23 Our current report adds to this literature by describing our learnings in engaging clinical and patient stakeholders to inform pain management in the primary care setting.

CONCLUSION

Engaging patients, clinicians, and other stakeholders as partners in comparative clinical effectiveness research may enhance the relevance of such studies, but unique challenges exist. Although researchers can meaningfully engage patients and other stakeholders, we contend that these processes must be coordinated with broader health care system supports if full implementation of the suggested patient-centered care initiatives is to occur. This study illustrates how patient-centered outcomes research might best maximize the benefits of stakeholder engagement and the potential effects of such input on clinical research. It is critical that we continue to improve our understanding and application of stakeholder engagement to develop, to fund, and to implement a research agenda that can maximally influence the care we deliver to patients with chronic health conditions.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Acknowledgements

The authors gratefully acknowledge that this study received funding support from the Patient-Centered Outcomes Research Institute and the National Institutes of Health, Center for Complementary and Integrative Health (R01 AT005896).

Brenda Moss Feinberg, ELS, provided editorial assistance.

How to Cite this Article

Elder CR, DeBar LL, Ritenbaugh C, et al. Health care systems support to enhance patient-centered care: Lessons from a primary care-based chronic pain management initiative. Perm J 2017;21:16-101. DOI: https://doi.org/10.7812/TPP/16-101.

References
1.    Ward BW, Schiller JS, Goodman RA. Multiple chronic conditions among US adults: A 2012 update. Prev Chronic Dis 2014 Apr 17;11:E62. DOI: https://doi.org/10.5888/pcd11.130389.
    2.    Centers for Disease Control and Prevention, Ambulatory and Hospital Care Statistics Branch. National ambulatory medical care survey: 2010 summary tables [Internet]. Atlanta, GA: Centers for Disease Control and Prevention; 2011 [cited 2014 Apr 10]. Available from: www.cdc.gov/nchs/data/ahcd/namcs_summary/2010_namcs_web_tables.pdf.
    3.    Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A. Improving chronic illness care: Translating evidence into action. Health Aff (Millwood) 2001 Nov-Dec;20(6):64-78. DOI: https://doi.org/10.1377/hlthaff.20.6.64.
    4.    Bodenheimer T, Chen E, Bennett HD. Confronting the growing burden of chronic disease: Can the US health care workforce do the job? Health Aff (Millwood) 2009 Jan-Feb;28(1):64-74. DOI: https://doi.org/10.1377/hlthaff.28.1.64.
    5.    Institute of Medicine. Relieving pain in America: A blueprint for transforming prevention, care, education, and research [Internet]. Washington, DC: Institute of Medicine of the National Academies; 2011 Jun. [cited 2016 Nov 25]. Available from: www.nationalacademies.org/hmd/~/media/Files/Report%20Files/2011/Relieving-Pain-in-America-A-Blueprint-for-Transforming-Prevention-Care-Education-Research/Pain%20Research%202011%20Report%20Brief.pdf.
    6.    Office of The Army Surgeon General, Pain Management Task Force. Pain management task force report: Providing a standardized DoD and VHA vision and approach to pain management to optimize the care for warriors and their families [Internet]. Falls Church, VA: Office of The Army Surgeon General; 2010 May [cited 2016 Nov 25]. Available from: www.regenesisbio.com/pdfs/journal/pain_management_task_force_report.pdf.
    7.    Mohr P, Deverka P, Monroe D, Mullins CD, Shalowitz M, Wallerstein N. Community-based participatory research: Lessons for stakeholder engagement in patient-centered outcomes research [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; 2016 [cited 2016 Sep 21]. Available from: http://effectivehealthcare.ahrq.gov/index.cfm/tools-and-resources/ehc-program-webcasts/community-based-research/transcript/.
    8.    Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA 2012 Apr 18;307(15):1587-8. DOI: https://doi.org/10.1001/jama.2012.442.
    9.    DeBar LL, Elder C, Ritenbaugh C, et al. Acupuncture and chiropractic care for chronic pain in an integrated health plan: A mixed methods study. BMC Complement Altern Med 2011 Nov 25;11:118. DOI: https://doi.org/10.1186/1472-6882-11-118.
    10.    Penney LS, Ritenbaugh C, Elder C, Schneider J, Deyo RA, DeBar LL. Primary care physicians, acupuncture and chiropractic clinicians, and chronic pain patients: A qualitative analysis of communication and care coordination patterns. BMC Complement Altern Med 2016 Jan 25;16:30. DOI: https://doi.org/10.1186/s12906-016-1005-4.
    11.    Paterson C, Britten N. In pursuit of patient-centred outcomes: A qualitative evaluation of the ‘Measure Yourself Medical Outcome Profile.’ J Health Serv Res Policy 2000 Jan;5(1):27-36. DOI: https://doi.org/10.1177/135581960000500108.
    12.    Paterson C. Seeking the patient’s perspective: A qualitative assessment of EuroQol, COOP-WONCA charts and MYMOP2. Qual Life Res 2004 Jun;13(5):871-81. DOI: https://doi.org/10.1023/b:qure.0000025586.51955.78.
    13.    QuickFacts: Portland city, Oregon [Internet]. Washington, DC: United States Census Bureau; 2010 [cited 2016 Sep 21]. Available from: www.census.gov/quickfacts/table/PST045215/4159000.
    14.    Ware JE Jr, Kosinski M, Bjorner JB, Turner-Bowker DM, Gandek B, Maruish ME. User’s manual for the SF-36v2 health survey. 2nd ed. Lincoln, RI: QualityMetric Incorporated; 2007.
    15.    IAP2. The IAP2 public participation spectrum [Internet]. Louisville, CO: International Association for Public Participation; 2000 [cited 2016 Feb 22]. Available from: http://c.ymcdn.com/sites/www.iap2.org/resource/resmgr/Foundations_Course/IAP2_P2_Spectrum.pdf?hhSearchTerms=%22public+and+participation+and+spectrum%22.
    16.    Patton MQ. Qualitative research and evaluation methods. 3rd ed. Thousand Oaks, CA: Sage Publications; 2002.
    17.    Bernard HR, Ryan GW. Analyzing qualitative data: Systematic approaches. Thousand Oaks, CA: Sage Publications; 2010.
    18.    Oliver S, Liabo K, Stewart R, Rees R. Public involvement in research: Making sense of the diversity. J Health Serv Res Policy 2015 Jan;20(1):45-51. DOI: https://doi.org/10.1177/1355819614551848.
    19.    Becker H, Stuifbergen A, Rogers S, Timmerman G. Goal attainment scaling to measure individual change in intervention studies. Nurs Res 2000 May-Jun;49(3):176-80. DOI: https://doi.org/10.1097/00006199-200005000-00011.
    20.    Forsythe LP, Ellis LE, Edmundson L, et. al. Patient and stakeholder engagement in the PCORI pilot projects: Description and lessons learned. J Gen Intern Med 2016 Jan;31(1):13-21. DOI: https://doi.org/10.1007/s11606-015-3450-z.
    21.    Forsythe LP, Frank L, Walker KO, et. al. Patient and clinician views on comparative effectiveness research and engagement in research. J Comp Eff Res 2015 Jan;4(1):11-25. DOI: https://doi.org/10.2217/cer.14.52.
    22.    Saunders T, Mackie TI, Shah S, Gooding H, de Ferranti SD, Leslie LK. Young adult and parent stakeholder perspectives on participation in patient-centered comparative effectiveness research. J Comp Eff Res 2016 Aug;5(5):487-97. DOI: https://doi.org/10.2217/cer-2016-0025.
    23.    Vandigo J, Oloyede E, Aly A, Laird AL, Cooke CE, Mullins CD. Continuous patient engagement in cardiovascular disease clinical comparative effectiveness research. Expert Rev Pharmacoecon Outcomes Res 2016;16(2):193-8. DOI: https://doi.org/10.1586/14737167.2016.1163222.

Subscriptions

Sponsored by the National Permanente Medical Groups, The Permanente Press publishes The Permanente Journal and books related to Kaiser Permanente and health care.

Circulation

27,000 print readers per quarter, 9,725 eTOC readers, and in 2016, 1.4 million page views on TPJ articles in PubMed from a broad international readership.

Letters

Articles, editorials, letters to the editor, and other material represent the opinion of the authors. Send your comments to permanente.journal@kp.org.


Copyright 2017 The Permanente Journal - Kaiser Permanente. All Rights Reserved.