Maintenance of Certification Part IV Quality-Improvement Project for Hypertension Control: A Preliminary Retrospective Analysis

Vallerie A Kolasinski, MPH, CHES; David W Price, MD, FAAFP

Perm J 2015 Spring; 19(2):36-40 [Full Citation]


Context: A Maintenance of Certification Part IV project was created on the basis of an existing, multifaceted hypertension improvement program.
Objective: To evaluate the impact of the Maintenance of Certification project, the effects of the improvement options on blood pressure control in hypertensive patients, and the participants' perception of the workload related to participation in the project.
Design: Nonexperimental retrospective analysis.
Setting: Kaiser Permanente hospitals and medical office buildings in Northern California.
Intervention: Participants used one or more options from a defined menu of strategies to attempt to increase the percentage of hypertensive patients on their patient panels who had controlled blood pressure.

Main Outcome Measure: Proportion of hypertensive patients with blood pressure ≤ 139/89 mm Hg.
Results: Fifty-two American Board of Family Medicine and 19 American Board of Internal Medicine certified physicians completed projects. Mean panel blood pressure control improved from 79.49% (standard deviation [SD] = 11.32) to 84.64% (SD = 7.80). The choice of improvement option was not associated with the level of improvement or with the participants' perception of the workload related to completing the project.
Conclusion: Project participants improved the care of their patients without an increased perceived burden to their practice. We found no association between the choice of improvement option and either the level of improvement or the perception of workload.


Approximately one-third of Americans have hypertension.1,2 If left unchecked, hypertension increases the risk of heart attack and stroke.2 Unfortunately, about half of those with hypertension have uncontrolled blood pressure.2 Because of the widespread public health impact of this condition, there have been many studies examining the effectiveness of quality-improvement (QI) programs focused on controlling blood pressure. A study conducted by Jaffe et al3 showed that a large-scale, multifaceted QI program focused on controlling hypertension can have a positive effect on blood pressure control.

QI aims to change aspects of the health care system to improve its efficiency and outcomes.4 Patient outcomes such as morbidity and mortality, recovery times, patient satisfaction, and costs are often used to evaluate QI programs.4,5

Performance-improvement strategies help individuals and teams work within a system to improve their own work processes. In health care, performance improvement is often used to reduce variation among physicians in the same field of practice.6 The American Medical Association process for performance improvement includes assessment of current practice, implementing a change in practice to address shortcomings found during the assessment phase, and evaluation of the changes implemented. Many frameworks for performance improvement, including Plan-Do-Study-Act and Lean Six Sigma, have been implemented in health care settings.7-9

Quality and performance improvement are incorporated into the continuous process of Maintenance of Certification (MOC) for American Board of Medical Specialties board-certified physicians. MOC is divided into four parts: professionalism, lifelong learning and self-assessment, cognitive expertise (passing a secure examination), and assessment of performance in practice.10 Quality and performance improvement are generally incorporated into this latter stage, MOC Part IV.11

In 2011, The Permanente Medical Group (TPMG), under the auspices of The Permanente Federation, joined the American Board of Medical Specialties Multi-Specialty MOC Portfolio Approval Program to grant MOC Part IV credit to diplomates for QI projects completed in Kaiser Permanente Northern California (KPNC).12 The program is composed of member boards from the American Board of Medical Specialties that recognize that institutional QI projects can satisfy the requirements for MOC Part IV.13

Under the program, participating organizations are allowed to designate QI projects for MOC Part IV credit. TPMG identified our ongoing hypertension QI program as a candidate for MOC credit because the program had been in place for several years, gave physicians access to aggregate panel data, and was a KPNC initiative. TPMG created an MOC Part IV project, called Blood Pressure Control for People with Hypertension, with the anticipated benefit of improving blood pressure control in participating physicians' patient panels by focusing on gaps in the individual participant's care of their patients with hypertension. When this project started in 2011, the proportion of hypertensive patients in KPNC with controlled blood pressure was 81%, which was short of the 84% target control rate set by TPMG but greater than the national rate of 47%.2 For TPMG physicians who completed this MOC Part IV QI project between September 2011 and November 2013, we evaluated the relationship between the improvement in each physician's patient panel, the types of interventions chosen to complete the project, and whether the type of intervention chosen affects the participant's perception of the workload that the project requires. We hypothesized: 1) there would be no statistically significant interaction between the physician's panel improvement and the type of intervention chosen to complete the project, and 2) the type of intervention chosen would not affect the physician's perception of workload.

The main outcome measure was the proportion of qualifying hypertensive patients with controlled blood pressure. A secondary outcome measure was the proportion of diabetic patients with controlled blood pressure. We also explored potential differences in the proportion of patients with controlled hypertension by the type of improvement option chosen; whether the number of improvement options completed had an effect on either the primary or secondary outcome measure; whether the type of intervention had an effect on the participant's perception of the increased burden of completing the MOC project; and whether the participant engaged in discussions with other members of their health care team about improving care of their hypertensive patients.

Participants were TPMG physicians certified by the American Board of Internal Medicine and/or the American Board of Family Medicine. E-mail messages about the opportunity to participate in the MOC project were sent to physicians who were participating in MOC and whose certification would expire within ten years from the date of sending the e-mail. In addition, presentations were made to family medicine and internal medicine leaders who were encouraged to disseminate information about the new MOC project. Participants were self-selected and completed the project as individuals or as part of a group with their colleagues.


This is a retrospective analysis of a QI program that was implemented through the Multi-Specialty MOC Portfolio Approval Program. This study was reviewed by the KPNC institutional review board and was designated as "not human subjects" research.

The QI project, described in the Introduction, took place across 22 KPNC Medical Centers and 48 medical offices.

To meet the qualifications of the Multi-Specialty MOC Portfolio Approval Program and to make the project more accessible to individual physicians, the improvement options from the original QI program were adapted so they could be implemented in a rapid-improvement cycle. The four improvement options were

1. employing tools, such as workflows, job aids, and patient education materials (Tools)

2. optimizing workflows with team members to take accurate blood pressure readings (Workflows)

3. reviewing evidence-based guidelines on appropriate care for hypertensive patients (Guidelines)

4. applying communication best practices to reinforce medication adherence or lifestyle changes (Communication).

Physicians commenced participation by visiting the project Web site, signing up for an account, and reading through the project description. The project description outlined the expected project length, eligible specialties, metrics, steps to participate in the project, group instructions, resources, how to fill out the attestation, and the project contact person for questions about the project. Physicians could choose to complete the project in a group with their colleagues or individually.

Participants were expected to work through the Plan-Do-Study-Act cycle, as outlined in the project description, to obtain a 1% to 2% increase in blood pressure control on their patient panel. In the Plan phase of the project, participants received monthly data that included the percentage of qualifying patients on their patient panel who had their blood pressure under control. Qualifying patients were determined on the basis of the Healthcare Effectiveness Data and Information Set (HEDIS) definition of patients between the ages of 18 and 85 years who had a diagnosis of hypertension (blood pressure ≥ 140/90 mm Hg) within the first 6 months of the year. Participants were expected to review their practices around that measurement and then select 1 or more improvement options to use to improve the rate of control among the qualifying patients on their panel.

In the Do phase of the project, participants implemented the improvement option or options for at least two months. Participants in the Study phase reviewed the results of their improvement option or options and planned their next steps. In the Act phase, participants were expected to implement these next steps. Participants were encouraged to complete as many Plan-Do-Study-Act cycles as necessary to achieve improvement, but two cycles were sufficient to obtain credit for the project.

After completing the requisite Plan-Do-Study-Act cycles, participants submitted an online attestation to receive both MOC and continuing medical education credit. The attestation included documentation of the starting and ending blood pressure control rates for their patient panel; improvements implemented; starting date, ending date, and number of QI cycles completed; how the participant reflected on the data; the perception of the impact the project had on the participant's work; whether team changes were implemented; key learnings; things that could have gone better; barriers to implementation; and suggestions for improving the project. The TPMG MOC team reviewed completed projects on a monthly basis to ensure that participants correctly completed the project attestations and to help those participants who had problems filling out their attestations. Afterward, participants were awarded 20 American Medical Association Physician's Recognition Award Category 1 credits for Performance Improvement Continuing Medical Education from The Permanente Federation, which notified the participants' American Board of Medical Specialities boards of successful project completion for MOC Part IV credit.

Data available from the attestations included the percentage of the participant's impaneled patients with a BP ≤ 139/89 mm Hg, both at the beginning and at the end of the project; improvement options used in the project; the participant's perception of the impact the project had on his/her clinical and operational work; and a few open-ended reflection responses. Most data were captured in open-ended text fields, but the perception of the project's impact on workflow was reported on a scale of 1 to 4.

Participants were disqualified from the study if they did not report the primary metric, if they had incomplete attestations, or if their attestation was outside the date range that we were studying. Participants who incorrectly reported a rolling metric instead of a monthly metric were disqualified as outliers. Figure 1 shows the breakdown of included (n = 73) and excluded (n  = 20) physicians.

Percentage improvement was calculated by subtracting the beginning metric from the ending metric. Reported free-text improvement options were categorized by an initial reviewer (VK) into one of four categories: Tools, Workflows, Guidelines, or Communication. A second reviewer (DP) checked categorizations for accuracy and agreement. Participants reported using multiple improvement options.

Statistical analysis of the dataset included descriptive and inferential statistics. Descriptive statistics consisted of means, medians, ranges, and frequencies. Paired t tests were used for comparing the metric both at the beginning and at the end of the project to determine if there was a significant change in blood pressure control. Because of the smaller sample size of a subset of patients with Type 1 and Type 2 diabetes, the Wilcoxon signed-ranks test was completed to compare the beginning and ending metrics for this subset. Analysis of variance was performed for quantitative data that compared the percentage improvement in blood pressure control and the number of improvement options used, the types of improvement options completed, and the perception of the impact the project had on the physician's work. The c2 analysis of independence was used to analyze the interaction between team discussions and perception of workload.



Data were collected between September 2011 and November 2013. Project durations ranged between 2 and 15 months, with a mean length of 4.85 months (standard deviation [SD] = 2.85). Seventy-three physicians certified by the American Board of Family Medicine and American Board of Internal Medicine completed the project (Table 1). There was an increase in the overall metric from a mean of 79.49% (SD = 11.32) to 84.64% (SD = 7.80), as shown in Table 2. The difference between the 2 means is statistically significant at the 0.05 level (t = -6.83, degrees of freedom [df] = 72). Similarly, 23 participants who reported the diabetic subset of the metric improved from a mean of 80.48% (SD = 7.13) to 84.57% (SD = 6.26). The difference between the 2 means is statistically significant at the 0.05 level (Z = -2.95). These data are summarized in Table 2.

Physicians chose to implement between 1 and 4 improvement options, with those who reported 1 intervention showing the highest mean improvement (mean = 7.17, SD = 6.51, n = 21), although the differences among the means were not statistically significant (F3,69 = 1.78). These data, along with data on the improvement options used and project impact, are summarized in Table 3. Neither the type of improvement options used nor the impact of the participants' participation on their work was statistically significant. Participants who completed 1 or 2 improvement options were more likely to report that the project made their work somewhat easier (c2 = 20.84, df = 9). However, the relationship between the specific intervention option and impact on workflow was not statistically significant (c2 = 4.41, df = 9).
A summary of these data appears in Table 4. Finally, the relationship between the team discussions and perception of impact was not statistically significant at the 0.05 level (c2 = 2.19, df = 3). These data are summarized in Table 5.





As hypothesized, we found no statistically significant interaction between the type of intervention chosen and the resulting improvement; nor did the type of intervention affect the physician's perception of workload to complete the project. Combined with an overall improvement in hypertension control, the results would indicate that participation in MOC within a QI setting does not have a negative effect on physician participants.14-16

Although overall improvement was observed in this cohort, KPNC overall also showed improvement during that same timeframe, from 81% in mid-2011 to 86.7% at the end of 2013. It is probable that the observed increase in the MOC cohort may have been mirroring that of the entire area, especially since this project was based on previously implemented strategies.

At the same time, the hypertension project was designed to enable physicians to improve their practice and to meet their board certification requirements. Overall, participants showed improvement during the project, without increasing the perception of the burden of work, of which MOC is often accused. This project demonstrates that MOC Part IV projects based on a health system's existing QI program is an effective way to document the part of MOC involving practice performance assessment.

A strength of this project is that it was based on an ongoing QI program that is familiar to our physicians. In addition, this project provided a framework for physicians to examine and explicitly reflect on their own strengths and weaknesses through the Plan-Do-Study-Act cycle and to implement improvement options that were tailored to their own experience.

Limitations of this project include the lack of a control group, participant self-selection, unaudited self-reporting of organizationally generated metrics, and a small sample size. The lack of a control group prevents the results of this study from being compared with those who did not participate in the project to determine whether the improvement observed was significantly different than in those who did not participate. Unfortunately, limited access to data outside the project database, paired with deidentified data from the project, prevented us from being able to obtain data for a control group.

Although this project was widely advertised to all TPMG internal and family medicine board certified physicians, only 93 of the potential 3136 qualifying physicians chose to participate in the project. In addition, the volunteer participants may have been more likely than others to be amenable to implementing QI programs and thus may not be representative of a broader population of physicians. Also, participants could have reported higher percentages at the end of their projects for fear of repercussions from reporting lower scores, or they could have reported incorrect values, or transcribed the data incorrectly. Likewise, the observed improvement may have been a result of increased attention to patients rather than from a particular intervention used in the project. Finally, small sample sizes, particularly in the subset analyses, may have resulted in missing significant associations.


Participants in this MOC project improved the care of their patients without an increased perceived burden to their practice, regardless of the project improvements chosen. We conclude that participation in this MOC Part IV project did not have a negative effect on participants’ panels nor on participants’ perception of the work required to complete the project. Further studies should examine the effect of other organizational QI efforts for MOC Part IV on different conditions, in different practice settings, with larger samples of physicians from different specialties, and with control groups to help discern potential differences between participating and nonparticipating physicians.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.


The authors would like to acknowledge Robert Knight, PhD, for his help with statistical analysis.

Kathleen Louden, ELS, of Louden Health Communications provided editorial assistance.

 1.    Health, United States, 2013: with special feature on prescription drugs: Table 64 [Internet]. Hyattsville, MD: National Center for Health Statistics; 2014 May [cited 2014 Oct 15].

vailable from:
    2.    High blood pressure fact sheet [Internet]. Atlanta, GA: Centers for Disease Control and Prevention; 2014 Jan 3 [cited 2014 May 27]. Available from:
    3.    Jaffe MG, Lee GA, Young JD, Sidney S, Go AS. Improved blood pressure control associated with a large-scale hypertension program. JAMA 2013 Aug 21;310(7):699-705. DOI:
    4.    Øvretveit J, Gustafson D. Evaluation of quality improvement programmes. Qual Saf Health Care 2002 Sep;11(3):270-5. DOI:
    5.    Donabedian A. Evaluating the quality of medical care. 1966. Milbank Q 2005;83(4):691-729. DOI:
    6.    Selby JV, Schmittdiel JA, Lee J, et al. Meaningful variation in performance: what does variation in quality tell us about improving quality? Med Care 2010 Feb;48(2):133-9. DOI:
    7.    Berwick DM. Developing and testing changes in delivery of care. Ann Intern Med 1998 Apr 15;128(8):651-6. DOI:
    8.    Going lean in health care. IHI Innovation Series white paper [Internet]. Cambridge, MA: Institute for Healthcare Improvement; 2005 [cited 2014 Jan 23]. Available from:
    9.    Bandyopadhyay JK , Coppens K. Six sigma approach to healthcare quality and productivity management. International Journal of Productivity and Quality Management 2005 Dec 15;5(1):V1-12.
    10.    ABMS maintenance of certification [Internet]. Chicago, IL: American Board of Medical Specialties; 2014 [cited 2014 Aug 20]. Available from:
    11.    Maintenance of certification part IV practice performance assessment [Internet]. Chicago, IL:

merican Board of Medical Specialties; 2013 Feb 13 [updated 2014 Apr 29; cited 2014 Aug 20]. Available from:
    12.    Approved portfolio sponsors [Internet]. Chicago, IL:

merican Board of Medical Specialties, Multi-Specialty MOC Portfolio Approval Program; 2014 [cited 2014 Apr 18]. Available from:
    13.    About the Portfolio Program [Internet]. Chicago, IL:

merican Board of Medical Specialties, Multi-Specialty MOC Portfolio Approval Program;
2014 [cited 2014 Apr 18]. Available from:
    14.    Holmboe ES, Wang Y, Meehan TP, et al. Association between maintenance of certification examination scores and quality of care for Medicare beneficiaries. Arch Intern Med 2008 Jul 14;168(13):1396-403. DOI:
    15.    Simpkins J, Divine G, Wang M, Holmboe E, Pladevall M, Williams LK. Improving asthma care through recertification: a cluster randomized trial. Arch Intern Med 2007 Nov 12;167(20):2240-8.
    16.    Holmboe ES, Meehan TP, Lynn L, Doyle P, Sherwin T, Duffy FD. Promoting physicians' self-assessment and quality improvement: the ABIM diabetes practice improvement module. J Contin Educ Health Prof 2006 Spring;26(2):109-19. DOI:


Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.


2 million page views of TPJ articles in PubMed from a broad international readership.


Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, and CrossRef.




ISSN 1552-5775 Copyright © 2021

All Rights Reserved