Refining Reporting Mechanisms in Oregon’s Patient-Centered Primary Care Home Program to Improve Performance


Sherril Gelmon, DrPH; Billie Sandberg, PhD; Nicole Merrithew, MPH; Rebekah Bally, MPH

Perm J 2016 Summer;20(3):15-115 [Full Citation]
E-pub: 05/20/2016


Introduction: As part of its strategy to achieve the Triple Aim, the Oregon Health Authority implemented the Patient-Centered Primary Care Home (PCPCH) Program in 2009. In 2014, the program recognized more than 500 primary care practices and had become an essential component of Oregon’s strategy for transforming health services delivery. To assist the Oregon Health Authority with evaluating practices’ achievement of the PCPCH model along its 6 core attributes (access, accountability, comprehensive care, continuity, coordination, and person-centered care), the research team developed an innovative scoring method.
Objective: To develop a synthesized attribute scoring methodology and to apply it to evaluate practices’ performance overall and along individual attributes.
Methods: The method builds on earlier studies of the medical home and draws on data from Oregon’s PCPCH recognition application and a survey of recognized practices. Scores are reported for each practice on the program’s core attributes, with further analyses by geography, practice size, and ownership. The method was pilot-tested in 30 practices recognized under the 2011 PCPCH standards and adapted for potential use with the program’s 2014 standards in 400 practices.
Results: Initial results demonstrate that the scores are effective for reporting performance to key program stakeholders, including provider practices. The method enables stakeholders to compare results across similar practices and across the model’s core attributes. The scores help analyze practice transformation over time, enabling practices and the Oregon Health Authority to identify opportunities for improvement and technical assistance.
Conclusions: The PCPCH Program could be replicated in other states. This article offers insights on implementation strategies, efficacy of the PCPCH model, and lessons learned.


During the past decade, interest in patient-centered medical home (PCMH) models has increased dramatically among policymakers, administrators, insurers, and others concerned with the quality and effectiveness of primary care delivery, especially to vulnerable populations. This article seeks to contribute to the knowledge base regarding the development and implementation of PCMH models by describing the implementation of Oregon’s Patient-Centered Primary Care Home (PCPCH) Program. Specifically, we recount the development of a unique attribute scoring method that was developed to help describe differences in characteristics and capabilities of primary care practices, and the results of the preliminary application of this method. We begin by providing an overview of Oregon’s PCPCH Program, its development, and the attributes that define this model of PCMH. A detailed overview of the attribute scoring method follows, including a description of the process by which the research team developed it. Next, there is a discussion of the results of the attribute scoring. Practices’ overall performance is examined, as well as performance along individual attributes and key practice characteristics. The article concludes with lessons learned from the research process and broader implications for use and replication of the method. This is intended as a descriptive report based on the Oregon experience; comparisons with other models and methods are beyond the scope of this article.

Oregon’s PCPCH Program is distinct from other PCMH models in the US; understanding the distinctions among these models is important to comprehend the methods employed here. The following section provides a general background on PCMH principles, an overview of a commonly recognized national model, and a description of the Oregon-specific model.

Patient-Centered Medical Home Model

The American Academy of Pediatrics initially developed the PCMH model in the 1960s to focus on providing both a central source of care and comprehensive medical records for children with special health care needs.1 Since that time, multiple versions of the medical home have expanded the scope to include general primary care across all age groups, yet they differ in their definitions and measurements of primary care processes, outcomes, and experience.2 Common principles across definitions and programs include the following: wide-ranging, team-based care; patient-centered orientation toward the whole person; care that is coordinated across all elements of the health care system and the patient’s community; enhanced access to care that uses alternative methods of communication; and a systems-based approach to quality and safety.3 The differences across models are evident in the specific requirements that practices must meet to demonstrate their attainment of a specific model.

The PCMH model promulgated by the National Committee for Quality Assurance (NCQA) is one of the most widely recognized medical home models. Conceptually, NCQA and Oregon’s PCPCH model are similar, but the actual elements within core attributes or requirements may be different, leading to a different emphasis in the evaluation of a clinic. For example, the NCQA model’s requirements to demonstrate the team-based care continuity principle include that clinics assist patients and families to select a personal clinician and document the selection in practice records; monitor the percentage of patient visits with a selected clinician or team; have a process to orient new patients to the practice; and collaborate with the patient and family to develop and implement a written care plan for patients transitioning from pediatric care to adult care.4 In contrast, the Oregon PCPCH model’s requirements for team-based care continuity include that clinics report the percentage of active patients assigned to a personal clinician or team and the percentage of patient visits with the assigned clinician or team. An additional commonality that links these two as well as other models is the overarching goal to improve health outcomes and the patient experience while decreasing unnecessary utilization and costs.1

Oregon’s Patient-Centered Primary Care Home Program

Oregon’s PCPCH Program, established by the state legislature in 2009, is viewed as a key strategy in achieving the “Triple Aim” envisioned in Oregon’s health systems transformation: improve the lifelong health of all Oregonians; increase the quality, reliability, and availability of care for all Oregonians; and lower or contain the cost of care so it is affordable for everyone.5 The PCPCH Program was established on the basis of evidence demonstrating that the medical home model is uniquely positioned to provide coordinated care and to advocate effectively for patients’ needs.6 The program is administered by the Oregon Health Authority (OHA) in Salem. It has the task of 1) developing strategies to identify and to qualify practices for the PCPCH Program; 2) using these same strategies to measure the quality of designated PCPCH practices; 3) promoting the development of PCPCH practices; and 4) encouraging individuals who are covered by multiple OHA programs, including the Oregon Health Plan, to receive care in settings implementing the PCPCH model.

The standards and measures that the PCPCH Program uses to evaluate the services delivered by practices seeking PCPCH designation were developed from 2009 through 2011 by a 15-member advisory committee, composed of patients, clinicians, and health plan and purchasing representatives appointed by the OHA.7 Current standards are based on a framework of 6 core attributes and are intended to reflect the perspective of the patient (see Sidebar: Core Attributes of Oregon’s Patient-Centered Primary Care Home Program).


Whereas some attributes are based on widely used and/or nationally accepted metrics, some have been defined through the combined input of staff and stakeholders. Specifications can be found in the OHA’s PCPCH Program Technical Specifications and Reporting Guide.8

The OHA developed a process by which practices are officially recognized by the state as meeting the PCPCH standards. The recognition program commenced in 2011 under the initial standards; a revised set of standards that addressed deficiencies identified during the start-up years was implemented in 2014. To initiate the recognition process, practices complete an application in which they self-attest to meeting the standards. The process does not require a practice to attain the highest PCPCH standards to achieve initial recognition.8 By setting a minimum threshold, program leaders and other stakeholders hoped a larger population of practices would seek recognition and could then be supported through educational opportunities and technical assistance resources to advance along the practice transformation spectrum. The OHA intended to ultimately promote a more substantial population impact by affecting a smaller, more incremental change across a broader population, as opposed to a larger change across a narrower population.9

In a comparison of the OHA certification requirements with a recognition program such as the NCQA’s PCMH, the NCQA appears to be more prescriptive and requires a more rigorous application process. For example, the PCMH from the NCQA requires that each practice meet all criteria categorized as a Level 1 before moving on to Level 2, whereas Oregon’s PCPCH model allows practices to demonstrate performance at various levels on different attributes, as well as meeting selected “must-pass” standards.a A more detailed comparison with NCQA and other models is beyond the scope of this article.

The NCQA medical home also requires that substantial amounts of documentation and fees be submitted at the time of application, whereas the PCPCH application is free of charge and is based largely on self-attestation.5,9,10 These were intentional design features for the OHA during the process of PCPCH model development because several stakeholders reported that these features of the NCQA model posed major barriers to practice participation. Although the NCQA PCMH clearly has merit for some organizations, the OHA’s strategy reflects its choice of more inclusive engagement.

Patient-Centered Medical Home Evaluations

The breadth of meta-analyses and evaluations available in the literature demonstrates that the number of primary care practices that participate in some sort of PCMH program is growing.11,12 At the same time, there is debate regarding not only the elements that constitute a PCMH but also the modalities to evaluate successful implementation of the model. This is particularly true given the variation in detailed requirements as previously discussed.

A number of studies have evaluated existing methods and developed new tools for larger evaluations. These tools include surveying practices about the motivation for pursuing NCQA PCMH recognition,13 assessing transformation over time using the Center for Medical Home Improvement’s Medical Home Index,14 creating a self-assessment checklist to monitor transformation progress,15 and broadening the Consumer Assessment of Healthcare Providers and Systems surveys to include PCMH measures.16 The Medical Home Index contains similar principles of the overall medical home model as previously discussed, but specific measures included in the tool differ from both the NCQA and Oregon models. The Consumer Assessment of Healthcare Providers and Systems is a series of surveys designed to assess patient experience with the health care system and services received. Two studies conducted by Blue Cross Blue Shield of Michigan discuss the development of a unique self-assessment tool modeled on the NCQA national recognition standards, as well as its implementation, as more than 1000 primary care practices in Michigan were designated as medical homes.17-20 The Blue Cross Blue Shield of Michigan PCMH self-assessment study was found to be useful in measuring long-term transformation progress in general, but researchers warned against assuming its use as a “1-size-fits-all” measure, thus limiting broad replication.19pS81

Although recognizing the definitional issues surrounding the PCMH model and appropriate evaluation methods, programs and researchers across the country are seeking to evaluate the impact of implementing a medical home model. Unfortunately, most of these evaluations remain relatively small in scope by focusing on a particular aspect of implementation (eg, addition of a care coordinator) or on one subpopulation (eg, older adults),12,21 by studying the transformation of only a handful of practices,22 or by using a small sample of practices.1 All of these methods limit increased understanding of the efficacy of the full model.

There is a dearth of evidence reporting on the overall implementation of PCMH models and assessing efficacy of the models.12p88-89,23 The body of work described here fills a gap in the literature describing implementation of a robust PCMH model implemented on a statewide basis with more than 200 practices at the time of this evaluation. This evaluation used a large data set (approximately 240 data points per practice) to assess differences in implementation across core principles of PCMH according to practice characteristics. This additional insight on practice transformation capability should provide a foundational basis for PCMH model refinement and policy discussions.


The OHA contracted with researchers at Portland State University in Portland, OR, to conduct an initial evaluation of the implementation of the PCPCH Program.24 This initial evaluation noted the challenge of synthesizing the extensive amount of data available about each practice from its recognition application into usable information that could assist the OHA and identify exemplary practices for benchmarking and technical assistance. Subsequently, the OHA invited the research team to develop a method that would address this challenge.

In an earlier project for the Oregon Primary Care Association based in Portland,17 the lead researcher (SG) developed a scoring method that built on the work of Rittenhouse and her colleagues25 in assessing medical practices. This work was well received by members of the Oregon Primary Care Association (community health centers). In addition, it proved very useful for technical assistance and training to assist these centers in advancing their work of primary care redesign (before the OHA PCPCH Program launch). The current research team decided to build on this concept and create an attribute scoring method for the OHA and the PCPCH Program.

Data Sources

The research team constructed the attribute scoring method by drawing on 2 sources of data. The first source is the practices’ recognition applications. Applications are scored on the basis of a predetermined and consistently applied scoring system. More specific information about the recognition scoring will not be discussed here but can be found in the OHA’s Technical Assistance and Reporting Guidelines.8 There are 10 must-pass measures, which are assigned no point values, and 57 measures that are assigned point values. Each must-pass measure represents a core component of the primary care home model as determined by the PCPCH Advisory Counsel. For example, under Attribute 3, comprehensive whole-person care, a practice is required to have “a screening strategy for mental health, substance use, or developmental conditions and documents on-site and local referral resources.” Staff members of PCPCH designate a practice as belonging to a particular “tier” depending on total points in their attestation application.26 A practice may achieve up to 380 points in the 2014 scoring system.

The second source of data is a pair of surveys conducted by the Portland State University research team in 2012 and 2013. These surveys were part of the first phase of evaluation of PCPCH implementation for the OHA using the 2011 standards. The PCPCH Program staff reviewed the evidence on best practices of medical home implementation and concluded that additional information would help the program to better understand various aspects of PCPCH practices and their implementation of the PCPCH model. The surveys addressed topics that were not included in the PCPCH application because the scope was beyond the program standards at the time. For example, the surveys asked questions about whether the PCPCH Program was advancing them toward accomplishment of the elements of the Triple Aim. Eighty-eight percent of the then-recognized PCPCH practices completed the first survey in 2012, and 69% completed the 2013 survey. These high response rates gave the research team and PCPCH Program staff confidence that this data set provided representative information about the practices.27 Several content areas that were addressed in the supplemental survey were integrated into the 2014 revision of the PCPCH recognition standards; other topics were added as optional questions in the 2014 recognition application.

The PCPCH application process and the supplemental survey generated a great deal of data about the recognized PCPCH practices. In an effort to synthesize these data to make it meaningful and useful to the OHA to understand various elements of the implementation of the PCPCH model, the research team developed an attribute scoring method. These scores, comparable to an index scoring method, may be used to compile multiple sets of information for a number of reasons.27 First, this method summarizes several variables into a single score, which reduces the difficulties of understanding complex data sets. Second, the quantitative measurements obtained with a summative score are amenable to analysis that is more precise. Finally, use of these attribute scores increases the reliability of the measurement itself because it is based on responses to more than a single question or item. The attribute scores are a measure of how a set of variables changes over time, and these scores give a simple overall picture of performance.

There is no set method or formula for developing such a scoring system, but certain concepts apply to all summary scoring methods.28 The most important of these is that the summary score is designed for a particular purpose and the design process involves choosing appropriate indicators, which are combined in a manner that supports the overall purpose of the summary score. The attribute scoring method developed for Oregon’s program was specifically designed to provide a comprehensive snapshot of the performance of recognized practices across multiple variables gleaned from two large data sets, and it drew on existing scoring approaches developed to measure primary care home infrastructure17 and large-scale implementation of the model.17


The attribute scoring method was created and scored using the 6 core attributes of the PCPCH Program, as well as its implementation, using the 2011 program standards criteria as well as questions posed in the 2012 and 2013 supplemental surveys conducted by Portland State University. Tables 1 through 7 provide the details of the attribute scoring method. Each Table delineates the elements that, when assessed collectively, comprise the total score available for each attribute. Each Table also distinguishes the source of the data by which the practices were scored. “Standard” refers to the questions that practices are required to answer in the PCPCH recognition application related to the recognition standards, and “optional” refers to the questions included in the PCPCH recognition application that are evidence-based but not specifically standard-derived. Some of these optional data were gleaned from the Portland State University supplemental survey because many practices opted not to answer these optional questions in 2014. This scoring builds on the already-established PCPCH scoring as feasible; for example, where the PCPCH application scores 5-10-15, this method scores the same item as 1-2-3. Scores were not weighted on the basis of whether the measure originated from the recognition application or the evaluation surveys. Because there is additional information provided from the optional, supplemental information, the scoring described here provides a broader picture of performance, but it is not intended to replace the scoring used by the PCPCH Program to assign practices to tiers in the recognition process.

Pilot Test

The attribute scoring method was pilot-tested in early 2014 with a sample of 30 recognized PCPCH practices to assess its viability. The pilot test provided several important findings. First, it demonstrated that the attribute scores tracked practices’ PCPCH application scores very closely for most practices in the sample. When practices’ attribute scores diverged substantially from their application scores, they generally scored lower on items included that originated from the supplemental survey (ie, not the existing standards). This preliminary result indicates that the inclusion of the optional information from the PCPCH applications is important for understanding practice performance. Second, individual practice characteristics, such as geographic location and organizational structure, seemed to play an important part in practices’ ability to perform at high levels in the pilot test. For example, large urban practices performed best overall, whereas small urban practices seemed to need the most technical assistance. Similarly, practices owned by larger systems seemed to perform at higher levels than those that are independent and unaffiliated.

The research team and OHA deemed these insights and trends in the data as important enough to warrant additional exploration.

Study Population

At the time of the study, there were 513 practices recognized as PCPCHs. Seventy-eight percent of these practices (N = 400) were recognized under the 2014 standards as of October 1, 2014; these practices comprise our study population.

The 2014 PCPCH recognition application includes required questions related to the recognition standards, as well as optional questions. The attribute scoring method uses data gleaned from practices’ answers to the required questions and to some of the optional questions. Many practices chose not to answer a number of the optional questions. Because the research team had access to the data from the 2013 supplemental survey that included identical information to the optional questions, a decision was made in consultation with PCPCH Program staff to augment the data files as appropriate to expand the total information available about each practice. This augmentation enhanced the total attribute scores created for each practice. The research team created reports on the basis of both raw scores and augmented scores to gauge the reliability of using the augmented scoring approach.









When the results were reviewed with PCPCH personnel, the conclusion was that the augmented data greatly improved the reliability of the scoring overall.

Total Scores

Figures 1 and 2 present the total attribute scores achieved on a scale of 0 to 100 for the 400 practices reviewed. Figure 1 displays the total attribute scores sorted, first, by standard question score (the required questions in the 2014 application, designated at bottom in dark grey) and then by optional question scores (at top in light grey). The standard question scores are the scores that are derived from those items in the recognition standards from the application; the optional question scores reflect the optional questions tested in the 2012 and 2013 Portland State University supplemental surveys that were then added to the 2014 recognition application. The average for the total scores is represented by the top solid line: 51.08 of 100 possible points. The average for the standard, required question scores is represented by the bottom line: 33.25 of 60 possible points.

For better understanding of the average total score, Figure 2 shows the distribution of total attribute scores received by recognized practices. Although the range of total scores is quite large, the most common scores that practices received were 58/100, 59/100, 49/100, 52/100, and 24/100. This distinction is important because it reveals that the average alone may be misleading because of the breadth in the range of scores. As shown, there is often a substantial variation in performance, which is obscured when one considers only the averages.

The data presented in Figure 1 demonstrate that there is more variation in the total attribute scores than in the scores based solely on the standard, required questions. This suggests that including the information from the optional questions matters in terms of better understanding how the PCPCH model is being implemented across Oregon. A fuller picture of what it means to be a PCPCH is provided when viewed through a framework of both required and optional information.

Individual Attribute Scores

The data presented indicate how practices scored on each of the six core attributes of the PCPCH Program, as well as on implementation of the model. Specifically, these data illustrate the extent to which the recognized practices achieved the PCPCH model along the continuum of the six core attributes and implementation.

Table 8 summarizes the total and standard scores for each attribute for all 400 practices. In addition to the actual point scores, the percentage of the average total score is presented. This percentage enables comparisons across standardized measures because each attribute has different total scores, and it better illustrates small differences in performance across each attribute. Sorting and displaying the attribute scores in this manner further demonstrates that the addition of the optional data makes a difference in illustrating the performance of the recognized practices.

When assessed on the total scoring criteria (standard plus optional information), the average scores are consistently lower than the average standard scores across all attributes. When the averages between the total attribute scores and the standard scores are compared, there are sizable differences in performance regarding Attributes 4 and 5 (continuity and coordination and integration, respectively). Further future analysis of the specific questions that contribute to these differences could reveal the source of this discrepancy. In addition, although most practices are categorized in the top tier in the 3-tier scoring system used by the PCPCH Program, the percentage scores reveal considerable potential for improvement in performance when considering all elements of each attribute.

Displaying the data in this manner demonstrates that the proportions of the averages seem to be fairly consistent (aside from the 2 attributes just mentioned). For example, if the average is 5/10 points for a standard question, then the average is 8/16 points for the total attribute score. This seems to suggest that, if a practice did well at implementing the requirements of the PCPCH Program, it also did well with the optional questions on Attributes 1, 2, 3, and 6.

Attribute Scores and Key Practice Characteristics

For further analysis of practices’ performance along the six core attributes of the PCPCH model as well as their implementation of the model, additional analysis focused on three key practice demographics: geographic location of the practice, practice ownership, and practice size. These three descriptive variables were selected because they proved to be important variables in the implementation of the PCPCH model, as demonstrated by the 2012 and 2013 supplemental surveys conducted by Portland State University.

Geographic Location

Oregon has considerable variation in population density; as a result, any statewide service delivery program must be attentive to the variable impact of urban or rural location on how a program is implemented. Categorizations of urban or rural location on the basis of population density were developed for the Portland State University supplemental surveys using classification systems employed by the Portland State University Population Research Center and the Oregon Office of Rural Health, Portland. All PCPCH practices were categorized on the basis of ZIP codes and US Census data to define geographic regions that are urban large (urbanized areas with greater than 200,000 population), urban medium (urbanized areas of 100,001 to 200,000), urban small (urbanized areas of 40,001 to 100,000), and rural (communities 10 or more miles from a population center of at least 40,000). None of the recognized practices exists in frontier regions (counties with fewer than 6 people per square mile). This categorization was compared with population density as a basis for categorization, and little difference was found. As a result, some locations are adjacent to a large metropolitan area but may be categorized as urban medium or small because their specific location is outside defined urban boundaries.

The results of this categorization defined 133 practices as rural; 98, as urban small; 60, as urban medium; and 109, as urban large. Figure 3 shows the overall average total attribute score across the 4 geographic categories is 51.08/100. There is little variation among the 4 groups, suggesting that geographic location is not a substantial factor in practices’ success with implementation of a PCPCH. On the basis of average total scores alone, practices in urban-medium settings performed best and practices in urban-large settings performed less well across all attributes. There is no clear explanation for this finding, and geographic location alone does not appear to explain the variation.

Organization Structure

The degree of autonomy or system affiliation of a practice was investigated because many of the practices are affiliated with large health services delivery systems, and others are independent. Several authors suggest the importance of organizational infrastructure in PCMH implementation.29-31 Health systems in Oregon generally have implemented a PCPCH with a single approach across all affiliated sites and have provided centralized resources to help individual sites implement PCPCH—a benefit that independent practices do not have. Practices were asked to describe their organization structure in terms of practice ownership and independence, identifying as one of the following: independent and unaffiliated with any other practice; independent governance but part of an alliance for shared group purchasing/other economies of scale; or owned by a larger system that governs the practice and determines operations, finances, and so on. Of the 400 practices, 131 were independent and unaffiliated, 60 were independently governed but part of an alliance, and 209 were owned by a larger system.

As demonstrated in Figure 4, there is no substantive difference observed in practices’ scores when categorized according to organizational structure. However, for 4 of the 6 attributes (Attributes 1, 3, 4, and 5), independent and unaffiliated practices demonstrate the highest scores. This might suggest that freedom from bureaucracy and standardization can more easily generate innovation. For Attribute 2 (accountability), practices owned by a larger system seem to perform substantially better. As stated previously, practices seem to struggle with demonstrating this concept, and the lowest overall scores for all practices across all 3 categories are for Attribute 2. In future, it may be worth exploring why larger systems seem to engender more accountability among their primary care practices.

Practice Size

The third organizational characteristic considered was practice size. There is some evidence that suggests that small practices face more challenges in becoming medical homes.17,21,32,33 Practices were asked to identify the number of full-time equivalent (FTE) primary providers (MD or DO, naturopathic doctor, nurse practitioner, or physician assistant) in their practice; these were categorized into 4 categories of 0 to 2, 3 to 5, 6 to 9, and 10 or more FTE primary providers. On the basis of self-reported data from the 400 practices, there were 89 practices with 0 to 2 FTE primary providers, 144 practices with 3 to 5 FTE primary providers, 81 practices with 6 to 9 FTE primary providers, and 86 practices with 10 or more FTE primary providers.

There is demonstrated variation across practices’ average scores, by practice size. Although the smallest practices have the lowest scores across all attributes, practices with 6 to 10 or more FTE clinicians have the highest scores overall. This may suggest that larger practices are better equipped to implement the PCPCH model because of more resources and capacity.32








This study has several limitations, primarily with regard to the data used. One potential limitation is the utilization of two sources of data to construct the attribute scores. The challenge was to blend the restricted content presented by the administrative data with survey data that are broader in scope. However, we view drawing on data from the surveys of practices to augment their recognition application data as advantageous, for it allows us to produce new information and insights as described previously. The predetermined variables included in the administrative data limited the scope of assessment efforts, which led to the decision to include the survey data. Additionally, we did not deem the discrepancy in dates (ie, that the survey data preceded the application data in time) of the data collection as important enough to warrant notation. Next, all data used here, whether gleaned from the recognition application or the supplemental survey, are self-reported by the practices. If the practices did not answer questions posed in the application or in the supplemental survey, then that information is not known and, thus, cannot be reported here. It is beyond the scope of this article to address in detail the issues involved in self-reporting, but we note that self-reporting limits the objectivity of the data. However, as we discussed earlier, in our pilot test we included only practices that had site visits to validate their application scores, and the initial results of the attribute scoring method followed these practices’ application scores closely.

Several other limitations should be noted. We did not validate our attribute scoring method against other like methods. As we note in the Introduction section of this article, the Oregon PCPCH Program differs enough from other recognition programs (eg, NCQA) that comparison of the attribute scoring method presented here with the method of other programs would be a disingenuous representation of the Oregon program and of our efforts. Sets of indicators such as our attribute scoring method are not a replacement for full-fledged program evaluation efforts to assess the outcomes and impact of this or any other medical home program. The results presented here are one part of larger evaluation efforts currently under way in Oregon to assess the PCPCH Program and its impact on population health, costs, and the patient experience.

Lessons Learned and Implications

The insights gained through the development of this attribute scoring method are valuable for the ongoing evaluation of PCPCH implementation in Oregon and for comparable evaluations of medical home models elsewhere. The results of our analysis demonstrate that the attribute scoring method is an effective tool for synthesizing a great deal of data on PCPCH practices and can be used to effectively assess their performance. This strategy could be replicated in other jurisdictions, and it offers insights on implementation strategies and the efficacy of PCMH models.

Specifically for Oregon’s PCPCH Program, the analysis has demonstrated that including the optional criteria from the PCPCH application has a substantial effect on understanding the performance of recognized practices, especially as it relates to Attribute 4, continuity, and Attribute 5, coordination and integration. This is consistently demonstrated when viewing average scores for all practices and across the 3 categorizations used for analysis (ie, geographic location, practice size, and organization structure). Integration of the required information, driven by a state health policy process and agreed on through a rigorous process of deliberation and consensus, with optional information identified by staff as reflecting best practices elsewhere, revealed a dilemma. The program sought parsimony in application (ie, a reasonable set of requirements for each practice to document as it applies for PCPCH recognition), but the research highlighted the breadth of knowledge and application identified through a comprehensive review of other medical home models. As has been stated previously, OHA’s strategy is inclusion rather than exclusion, so it was intentional in not creating an overly burdensome process that would be a barrier, in particular to smaller, nonurban practices across the state.

Specific criteria such as geographic location, practice size, and organization structure do not seem to play a substantial role in practices’ performance related to the implementation of the PCPCH Program. However, there are a number of discernible patterns in the data across these categorizations that warrant further study and analysis, and could provide program leadership with some direction for future technical assistance efforts and for identification of exemplary practices from which others can learn. Additionally, analyses of the results based on these 3 descriptors may help the PCPCH Program in responding to questions and challenges from various stakeholder groups. Although there is substantial variation in practices’ performance along each attribute, performance related to the implementation of Attribute 2, accountability, is consistently low across all categorizations and in comparison with the other 5 attributes. This could merit further investigation in the future, especially given the large number of practices that are part of a health system and thus potentially have less autonomy in how PCPCH is implemented at each site.

Although the method described here will not necessarily be directly applicable to other PCMH models that use different categorizations or scoring systems, this approach does offer a strategy that could be adapted to distill large amounts of data into a small number of meaningful descriptors that will be useful to both program administrators and participants. Depending on the data available, other analyses could investigate performance on the basis of categorizations by patient population demographics, specific disease conditions, provider specialty, or other factors deemed important and reported by the practices.


The next steps for evaluation of PCPCH implementation in Oregon include building on the attribute scoring, identifying exemplary practices, and conducting in-depth case studies to better understand the facilitators of, and barriers to, successful implementation of a PCMH model of service delivery. The PCPCH Program intends to use a combination of this information to inform the next revision of the PCPCH standards, as well as any lessons that could be gleaned for development of a behavioral health home model and/or program. The PCPCH Program convened a broad stakeholder group through the end of 2015 to discuss specifications for the revision of standards and measures.

a The OHA staff has created several internal working documents that compare NCQA and PCPCH requirements at a detailed level. Although this information is not readily publicly available, additional information can be supplied on request.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

No internal review board approval was sought because the work did not involve human subjects.

The project also was supported in part by Funding Opportunity Number CMS-1G1-12-001 from the US Department of Health and Human Services, Centers for Medicare and Medicaid Services, Baltimore, MD. The content provided is solely the responsibility of the authors and does not necessarily represent the official views of the US Department of Health and Human Services, the Oregon Health Authority, or any of its agencies.


The authors wish to acknowledge the support of the Oregon Health Authority, Salem, in funding this research, as well as the collaboration of staff working in the Office of the Patient-Centered Primary Care Home Program.

Kathleen Louden, ELS, of Louden Health Communications provided editorial assistance.

How to Cite this Article

Gelman S, Sandberg B, Merrthew N, Bally R. Refining reporting mechanisms in Oregon’s patient-centered primary care home program to improve performance. Perm J 2016 Sum;20(3):15-115. DOI:

1.    Burton RA, Devers KJ, Berenson RA. Patient-centered medical home recognition tools: a comparison of ten surveys’ content and operational details [Internet]. Washington, DC: The Urban Institute, Health Policy Center; 2011 May [cited 2015 Dec 29]. Available from:
    2.    Robert Graham Center. The patient-centered medical home: history, seven core features, evidence and transformational change [Internet]. Washington, DC: Robert Graham Center, Center for Policy Studies in Family Medicine and Primary Care; 2007 Nov [cited 2015 Dec 29]. Available from:
    3.    Jackson GL, Powers BJ, Chatterjee R, et al. The patient-centered medical home: a systematic review. Ann Intern Med 2013 Feb 5;158(3):169-78. DOI:
    4.    National Committee for Quality Assurance. PCMH 2011-PCMH 2014 crosswalk [Internet]. Washington, DC: National Committee for Quality Assurance; 2015 Oct [cited 2015 Dec 29]. Available from:
    5.    Oregon Health Authority. Patient-centered primary care home program [Internet]. Salem, OR:; [cited 2015 Dec 29]. Available from:
    6.    Oregon Health Authority. Oregon’s action plan for health [Internet]. Salem, OR: Oregon Health Authority; 2010 Dec [cited 2015 Dec 29]. Available from:
    7.    Office for Oregon Health Policy and Research. Standards and measures for patient-centered primary care homes: final report of the patient-centered primary care home standards advisory committee. Salem, OR: Oregon Health Authority; 2012 Oct.
    8.    Oregon Health Authority. Oregon Health Authority patient-centered primary care home program 2014 recognition criteria: technical specifications and reporting guide [Internet]. Salem, OR: Oregon Health Authority; updated 2013 Oct 4 [cited 2015 Dec 29]. Available from:
    9.    Paustian ML, Alexander JA, El Reda DK, Wise CG, Green LA, Fetters MD. Partial and incremental PCMH practice transformation: implications for quality and costs. Health Serv Res 2014 Feb;49(1):52-74. DOI:
    10.    Laramie A. The PCMH recognition process: understanding the application, scoring and recognition process [Internet]. Boulder, CO: Coleman Associates; [cited 2014 May 2]. Available from:
    11.    Zutshi A, Peikes D, Smith K, et al; Mathematica Policy Research. The medical home: what do we know, what do we need to know? A review of the earliest evidence on the effectiveness of the patient-centered medical home model. AHRQ pub no. 12(14)-0020-1-EF [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; 2013 Mar [cited 2015 Dec 29]. Available from:
    12.    Williams JW, Jackson GL, Powers BJ, et al. Closing the quality gap: revisiting the state of the science (vol. 2: the patient-centered medical home). Evid Rep Technol Assess (Full Rep) 2012 Jul;(208.2):1-210.
    13.    Hahn KA, Gonzalez MM, Etz RS, Crabtree BF.
National Committee for Quality Assurance (NCQA) patient-centered medical home (PCMH) recognition is suboptimal even among innovative primary care practices. J Am Board Fam Med 2014 May-Jun;27(3):312-3. DOI:
    14.    McAllister JW, Cooley WC, Van Cleave J, Boudreau AA, Kuhlthau K. Medical home transformation in pediatric primary care—what drives change? Ann Fam Med 2013 May-Jun;11 Suppl 1:590-8. DOI:
    15.    Daniel DM, Wagner EH, Coleman K, et al. Assessing progress toward becoming a patient-centered medical home: an assessment tool for practice transformation. Health Serv Res 2013 Dec;48(6 Pt 1):1879-97. DOI:
    16.    Scholle SH, Vuong O, Ding L, et al.
Development of and field test results for the CAHPS PCMH Survey. Med Care 2012 Nov; 50 Suppl:S2-10. DOI:
    17.    Rittenhouse DR, Casalino LP, Gillies RR, Shortell SM, Lau B. Measuring the medical home infrastructure in large medical groups. Health Aff (Millwood) 2008 Sep-Oct;27(5):1246-58. DOI:
    18.    Berenson RA, Hammons T, Gans DN, et al. A house is not a home: keeping patients at the center of practice redesign. Health Aff (Millwood) 2008 Sep-Oct;27(5):1219-30. DOI:
    19.    Alexander JA, Paustian M, Wise CG, et al. Assessment and measurement of patient-centered medical home implementation: the BCBSM experience. Ann Fam Med 2013 May-Jun;11 Suppl 1:S74-81. DOI:
    20.    Annis Emeott A, Markovitz A, Mason MH, et al. Four-year evolution of a large, state-wide patient-centered medical home designation program in Michigan.
Med Care 2013 Sep;51(9):846-53. DOI: http://
    21.    Baxter L, Nash DB. Implementing the patient-centered medical home model for chronic disease care in small medical practices: practice group characteristics and physician understanding. Am J Med Qual 2013 Mar-Apr;28(2):113-9. DOI:
    22.    Wagner EH, Gupta R, Coleman K. Practice transformation in the safety net medical home initiative: a qualitative look. Med Care 2014 Nov;52(11 Suppl 4):S18-22. DOI:
    23.    Rissi JJ, Gelmon S, Saulino E, Merrithew N, Baker R, Hatcher P. Building the foundation for health system transformation: Oregon’s Patient-Centered Primary Care Home program. J Public Health Manag Pract 2015 Jan-Feb;21(1):34-41. DOI:
    24.    Ross D, Gelmon S. Primary Care Learning Community Assessment. Portland, OR: Oregon Primary Care Association; 2009.
    25.    Oregon Health Authority. Oregon patient-centered primary care home model: implementation reference guide [Internet]. Salem, OR:; 2011 [cited 2014 May 2]. Available from
    26.    Gelmon SB, Trotta R. The patient-centered primary care home (PCPCH). Executive summary: results of the 2012-2013 supplemental surveys [Internet]. Portland, OR: Portland State University; Oregon Health Authority; 2013 [cited 2014 May 2]. Available from:
    27.    Frankfort-Nachmias C, Nachmias D. Research methods in the social sciences. 6th ed. New York, NY: Worth Publishers, Inc; 2000.
    28.    Cooley WC, McAllister JW, Sherrieb K, Clark RE.
The medical home index: development and validation of a new practice-level measure of implementation of the medical home model. Ambul Pediatr 2003 Jul-Aug;3(4):173-80. DOI:<0173:tmhida>;2.
    29.    McNellis RJ, Genevro JL, Meyers DS. Lessons learned from the study of primary care transformation. Ann Fam Med 2013 May-Jun;11 Suppl 1:S1-5. DOI:
    30.    Pedersen S. Transformation evaluation of the Maryland multi-payor patient centered medical home program: initial site visits [Internet]. Presentation at AcademyHealth 2014 Annual Research Meeting; 2014 Jun 8-10; San Diego, CA. Columbia, MD: IMPAQ International, LLC; 2014 [cited 2015 Dec 29]. Available from:
    31.    Nutting PA, Crabtree BF, McDaniel RR. Small primary care practices face four hurdles—including a physician-centric mind-set—in becoming medical homes. Health Aff (Millwood) 2012 Nov;31(11):2417-22. DOI:
    32.    Rittenhouse DR, Casalino LP, Shortell SM, et al. Small and medium-size physician practices use few patient-centered medical home processes. Health Aff (Millwood) 2011 Aug;30(8):1575-84. DOI:
    33.    Scholle SH, Asche SE, Morton S, Solberg LI, Tirodkar MA, Jaén CR. Support and strategies for change among small patient-centered medical home practices. Ann Fam Med 2013 May-Jun;11 Suppl 1:S6-13. DOI:


Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.


2 million page views of TPJ articles in PubMed from a broad international readership.


Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, and CrossRef.




ISSN 1552-5775 Copyright © 2021

All Rights Reserved