A Commentary on “Comparative Health Systems Research Among Kaiser Permanente and Other Integrated Delivery Systems: A Systematic Literature Review”

Kaiser Permanente, The Permanente Medical Group, The Permanente Federation


Thomas E Kottke, MD, MSPH; Patrick T Courneya, MD


Perm J 2014 Summer;18(3):94-95 [Full Citation]


"[O]ccupational organization . . . constitutes a dimension quite as distinct and fully as important as its knowledge."1

In this issue of The Permanente Journal, Maeda et al2 enumerate the relative frequency at which articles on comparative health systems research are published about Kaiser Permanente (KP) and other integrated delivery systems. Searching PubMed and the KP Publications Library, they found that a mere 4% of publications met their criteria. This is unfortunate because well-organized systems are the foundation of effective health services delivery, and comparative health systems research to create these systems can greatly increase the value that Americans receive from their health care.

After observing that the physicians he studied tended to practice very much like their colleagues despite differences in training and upbringing, Eliot Freidson1 concluded that "occupational organization … constitutes a dimension quite as distinct and fully as important as its knowledge … ." Having practiced medicine at many different sites over our careers, we have found that the organization of care in a particular location determined the services that we could offer the patient, the efficiency with which we provided those services, and the patient's experience with the care that s/he received from us. Therefore, we would have to agree with Freidson—organizational context is a powerful determinant of clinical performance. The paucity of comparative health systems research relative to the overall investment in research represents a missed opportunity for research to contribute to the Triple Aim.

Despite contributing nearly one-third of the international investment in biomedical research and spending more on health care than any other country, Americans receive a poor return on their investment. Americans die sooner and experience more illness than do residents of many other countries.3,4 In 2007, life expectancy at birth for US males was 17th of 17 peer countries; life expectancy for US females was 16th of 17 countries.4 We believe that these poor results are due, in part, to the paucity of comparative organizational research aimed at improving outcomes for individual patients and entire populations.

This paucity also contributes to the persistence of inequitable care in America. Colorectal cancer screening rates are markedly lower for individuals of low economic and educational status.5 Additionally, in a large cohort of whites and African Americans, follow-up of colorectal abnormalities was significantly lower among the African Americans.6 Inequalities of care like these contribute to large differences in life expectancy, even in single metropolitan areas. In the Twin Cities, MN, for example, age-standardized mortality rates for American Indians are six times as high as they are for Asians.7 Life expectancy in some Twin Cities zip codes is equal to that of Armenia, Estonia, and Jordan, and it is exceeded by life expectancy in the Gaza Strip in the Middle East, in Romania, and in China.8

However, poor health outcomes and disparities do not need to be a part of American life. In a recent issue of The Permanente Journal, Tuso et al9 reported their analysis of the care processes that reduced the risk of readmission because of heart failure. They found that a complex case conference, along with a visit from a home health nurse and a follow-up visit with the patient's physician reduced the risk of readmission by about 45%. Among the 21 patients for whom complete data were available and who were treated around the time of the study, there were 81 admissions in the 6 months before and 22 admissions in the 6 months after the complex case management conference was implemented, a reduction of 68%.

HealthPartners in St Paul, MN, has used comparative health systems research to significantly reduce disparities in care while addressing screening targets for all patients. The tools that we have used include collecting language preference and racial identification from our patients, identifying and reporting the levels of care disparities, diagnosing the sources of the disparities, developing initiatives that address the particular causes of the disparities, and continuously monitoring the results with monthly report cards to assure ourselves that the new system is outperforming the prior system.

For example, the gap in mammography rates between white women and women of color was 12.3 percentage points in 2007, and appointment no-show rates for women of color were high. We met with women in the community and learned that returning to the clinic for a second visit was a major barrier to mammography. We instituted a same-day mammography program and were able to decrease the disparity rate to 6.1 points by 201010 and 3.2 points by the fourth quarter of 2013 (unpublished written data available on request). Likewise, by offering multiple options for colorectal cancer screening (eg, fecal immunochemical test or colonoscopy) and actively promoting screening to patients of color, we have been able to reduce the gap in colorectal cancer screening between white patients and patients of color from 26.2 points during the first quarter of 2009 to 12.0 points in July 2013 (unpublished written data available on request). While decreasing health disparities, we have also increased screening rates for both colorectal cancer and breast cancer for all groups of patients.

By using the organizational tools described earlier and employing an East African community health worker to follow-up with patients in the community, HealthPartners clinicians at one clinic were able to reduce the disparity in optimal diabetes care for patients of color to just 4.4 points in February 2014 (unpublished written data available on request). The rate of optimal diabetes care for patients of color—45.4%—is 7 points above the average for medical groups reporting to Minnesota Community Measurement, a nonprofit organization whose mission is to accelerate the improvement of health by publicly reporting health care information.11

We believe that creating an environment that comprises five activities has allowed us to achieve the goals described above: 1) a set of mutual, measurable goals; 2) public reporting of the extent to which the goals are being achieved; 3) sufficient resources to achieve the goals; 4) alignment of stakeholder incentives, imperatives, and sanctions with the goals; and 5) leadership among all stakeholders to endorse and promote the goals.12

In the past, the high cost of obtaining claims data and clinical data was a legitimate barrier to comparative health systems research. However, with the implementation of electronic billing and electronic health records, large group practices like KP, and medical group consortia such as the Health Maintenance Organization Research Network with its Virtual Data Warehouse,13 comparative health systems research is now economically feasible. Although the impact of participation in care improvement collaboratives is limited in many cases,14 we believe that participation in the Institute for HealthCare Improvement's programs helps health care organizations improve their care through self-evaluation and shared learning. To stimulate care improvement, HealthPartners has published clinic performance reports on selected indicators for many years.15 These reports have been associated with meaningful improvements in performance by clinical groups that treat HealthPartners members. Public reporting of health system performance by national organizations such as the Healthcare Effectiveness Data and Information Set (HEDIS),16 Hospital Compare,17 and the Leapfrog Group18 might also be expected to drive competition for outcomes improvement by groups that are not formally affiliated. Although improving monthly performance reports should be considered adequate evidence of goal attainment, a stepped-wedge randomized evaluation design would also be a powerful and efficient evaluation tool that, in most cases, would satisfy the exigent circumstances of care delivery system development.19

"Knowing is not enough; we must apply. Willing is not enough; we must do." This credo, attributed to Johann W von Goethe, appears on the frontispiece of all Institute of Medicine reports and underscores the fact that biomedical research and health care are social investments that are expected to create value for the public. More than 40 years ago, Freidson1 recognized that an effective care delivery system is the substrate that creates value from knowledge. Maeda et al2 have done the research documenting the paucity of research designed to develop this substrate. It is now time for the health care research and practice communities to collaborate, increase comparative health systems research activity, and give Americans the value in health care that they deserve.


Kathleen Louden, ELS, of Louden Health Communications provided editorial assistance.


   1.  Freidson E. Profession of medicine: a study of the sociology of applied knowledge. New York, NY: Dodd, Mead & Company; 1970. p xi.

   2.  Maeda JL, Lee K, Horberg M. Comparative health systems research among Kaiser Permanente and other integrated delivery systems: a systematic literature review. Perm J 2014 Summer;18(3):epub 2014 June 9. DOI: https://doi.org/10.7812/TPP/13-159.

   3.  Global competitiveness—the importance of US leadership in science and innovation for the future of our economy and our health [Internet]. Bethesda, MD: US Department of Health & Human Services, National Institutes of Health; 2012 [cited 2014 Apr 21]. Available from: www.nih.gov/about/impact/impact_global.pdf.

   4.  Woolf SH, Aron L, editors. US health in international perspective: shorter lives, poorer health. Washington, DC: The National Academies Press; 2013.

   5.  Steele CB, Rim SH, Joseph DA, King JB, Seeff LC; Centers for Disease Control and Prevention (CDC). Colorectal cancer incidence and screening—United States, 2008 and 2010. MMWR Surveill Summ 2013 Nov 22;62 Suppl 3:53-60.

   6.  Laiyemo AO, Doubeni C, Pinsky PF, et al. Race and colorectal cancer disparities: health-care utilization vs different cancer susceptibilities. J Natl Cancer Inst 2010 Apr 21;102(8):538-46. DOI: https://doi.org/10.1093/jnci/djq068.

   7.  Helmstetter C, Brower S, Egbert A. The unequal distribution of health in the Twin Cities. St Paul, MN: Amherst Wilder Foundation; 2010 Oct.

   8.  The world factbook. Country comparison: life expectancy at birth [Internet]. Washington, DC: Central Intelligence Agency; updated 2014 [cited 2013 Sep 7]. Available from: www.cia.gov/library/publications/the-world-factbook/rankorder/2102rank.html.

   9.  Tuso P, Watson HL, Garofalo-Wright L, et al. Complex case conferences associated with reduced hospital admissions for high-risk patients with multiple comorbidities. Perm J 2014 Winter;18(1):38-42. DOI: https://doi.org/10.7812/TPP/13-062.

10.  von Sternberg T, Averbeck B, McClure N. Collection of data on race and ethnic group by physician practices. N Engl J Med 2010 Jul 1;363(1):96; author reply 97. DOI: https://doi.org/10.1056/NEJMc1004036.

11.  Minnesota healthscores. Diabetes [Internet]. Minneapolis, MN: MN Community Measurement; updated 2013 Dec 13 [cited 2014 Mar 21]. Available from: www.mnhealthscores.org/index.php?p=group_reports&sf=group&search_phrase=&category_section=category_condition&category=1&name_id=&compare=.

12.  Kottke TE, Pronk NP, Isham GJ. The simple health system rules that create value. Prev Chronic Dis 2012;9:E49. DOI: https://doi.org/10.5888/pcd9.110179.

13.  HMORN Virtual Data Warehouse [Internet]. 2012; [cited 2014 Apr 23]. Available from: www.hmoresearchnetwork.org/en/Tools%20&%20Materials/VDW/.

14.  Schouten LM, Hulscher ME, van Everdingen JJ, Huijsman R, Grol RP. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 2008 Jun 28;336(7659):1491-4. DOI: https://doi.org/10.1136/bmj.39570.749884.BE.

15.  Wehrle D, Bussey S. 2012 clinical indicators report. Minneapolis, MN: HealthPartners; 2012.

16.  HEDIS and quality compass [Internet]. Washington, DC: National Committee for Quality Assurance; updated 2014 [cited 2014 Apr 14]. Available from: www.ncqa.org/HEDISQualityMeasurement/WhatisHEDIS.aspx.

17.  Hospital compare [Internet]. Baltimore, MD: Centers for Medicare & Medicaid Services; updated 2014 [cited 2013 Jan 31]. Available from: www.medicare.gov/HospitalCompare/.

18.  Pronovost P, Thompson DA, Holzmueller CG, Dorman T, Morlock LL. Impact of the Leapfrog Group's intensive care unit physician staffing standard. J Crit Care 2007 Jun;22(2):89-96. DOI: https://doi.org/10.1016/j.jcrc.2006.08.001.

19.  Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials 2007 Feb;28(2):182-91. DOI: https://doi.org/10.1016/j.cct.2006.05.007.



The Permanente Journal is celebrating it's 20th anniversary year. We look forward to continuing to bring you more high-quality content during the next 20 years.


The Permanente Journal (ISSN 1552-5767) is published quarterly by The Permanente Press. Journal subscriptions are entered for the calendar year. Advance payment in US dollars is required.


25,000 print readers per quarter, 7,628 eTOC readers, and in 2016, 1.4 million page views on TPJ articles in PubMed from a broad international readership.


Articles, editorials, letters to the editor, and other material represent the opinion of the authors. Send your comments to permanente.journal@kp.org.

Copyright 2017 The Permanente Journal - Kaiser Permanente. All Rights Reserved.