End-Stage Renal Disease Outcomes among the Kaiser Permanente Southern California Creatinine Safety Program (Creatinine SureNet): Opportunities to Reflect and Improve
Perm J 2017;21:16-143
Objectives: The Kaiser Permanente Southern California (KPSC) creatinine safety program (Creatinine SureNet) identifies and outreaches to thousands of people annually who may have had a missed diagnosis for chronic kidney disease (CKD). We sought to determine the value of this outpatient program and evaluate opportunities for improvement.
Screening for chronic kidney disease (CKD) remains controversial. Among high-risk populations, screening and surveillance for CKD is recommended.1-3 However, organizations such as the US Preventive Services Task Force and the American College of Physicians do not recommend routine screening for asymptomatic individuals in the general population.4,5 The method of assessing kidney function with estimated glomerular filtration rate (eGFR) derived from blood creatinine measurements has also been an area of ambiguity. The eGFR calculations all have inherent accuracy and reliability concerns.6 Among them, the Chronic Kidney Disease Epidemiology Collaboration equation (CKD-EPI) has been shown to have superior accuracy and prognostic value compared with others such as the Cockcroft-Gault and the Modification Diet in Renal Diseases (MDRD) equations.7,8 Another important consideration is the role of urine studies because they help to define and prognosticate CKD.9-12
CKD is highly prevalent and associated with adverse outcomes, including end-stage renal disease (ESRD), cardiovascular events, and all-cause mortality.8,13-15 Among adults in the US, the estimated prevalence is around 14%.13 Unfortunately, CKD is not always identified and managed in an optimal and timely manner.16,17
We previously described the Kaiser Permanente Southern California (KPSC) creatinine safety program (Creatinine SureNet), which was designed to identify and to reach potential patients with CKD who otherwise would have been missed.18 This program leveraged the health system and central laboratory data to create a surveillance system to identify and to outreach to more than 12,000 individuals who did not have a follow-up creatinine test after an initial abnormal creatinine measurement.
Our current study sought to determine the importance of the KPSC creatinine safety program and to identify areas for improvement. We sought to determine the overall rate of ESRD among this presumed high-risk creatinine safety program cohort compared with the rest of the KPSC population. We also evaluated how often patients with confirmed CKD received important complementary studies such as urine analyses and urine protein quantitation. Finally, we sought to compare the rate of capture into the KPSC creatinine safety program if the CKD-EPI had been used to calculate eGFR instead of the existing MDRD equation.
A longitudinal cohort study was performed from February 1, 2010, through December 31, 2015, among KPSC members with blood creatinine laboratory studies whose results indicated a reduced eGFR using the MDRD formula. KPSC is an integrated health system composed of 14 Medical Centers, more than 200 satellite clinics, and more than 6000 physicians who care for greater than 4 million members in Southern California. Information on demographics, laboratory results, comorbidities, and clinical events that were captured as part of routine clinical care were extracted from the electronic health records. All laboratory measurements were performed and reported from an American College of Pathology/Clinical Laboratory Improvement Act (CLIA) certified laboratory. Kidney function is reported in the electronic health record as eGFR, calculated using the modified 4-variable MDRD equation.19 For the purpose of our current study, we also calculated eGFR in the same individuals using the CKD-EPI equation.7 End-stage renal disease was defined as any individual receiving dialysis or who received a renal transplant. The study population was followed-up until they reached ESRD, died, or lost KPSC membership, or until the end of study observation (December 31, 2015). This study was approved by the local institutional review board and exempted from the need for informed consent (no. 10572).
The details of the KPSC creatinine safety program have been previously described.18 Born from the KPSC Complete Care program established in 2009, the creatinine safety program (called Creatinine SureNet) was one of many safety nets implemented to capture clinical care gaps using electronic health surveillance and multidisciplinary outreach.20 The Creatinine SureNet uses the concept of electronic clinical surveillance to identify an abnormal creatinine measurement that was not followed-up with a repeated measurement.21 Individuals with a single creatinine measurement that computed to an eGFR less than 60 mL/min (MDRD equation) and had no repeated eGFR measurement 90 days or more later were included. Individuals who fit the following formula—eGFR + ½ age > 85 years—were excluded.22 From February 1, 2010, to March 1, 2014, more than 12,000 members were identified by the Creatinine SureNet. A coordinated effort between a centralized regional nurse and clinicians was used to communicate with patients to obtain a second measurement.
Outcomes and Analyses
All individuals who were included in the Creatinine SureNet cohort were characterized by eGFR and whether they had a follow-up creatinine measurement or other urine tests. The primary outcome was incident ESRD. Age- and sex-adjusted ESRD incidence rates by year were determined for the SureNet cohort and for the KPSC general population for comparison. Age and sex adjustments were standardized to the US Census 2010 population.23 The relative risk (RR) of the SureNet cohort relative to the KPSC general population was estimated, for each year and for all years combined, as the ratio of the 2 incidence rates. A 2-sided test of the null hypothesis that RR = 1 was performed. All-cause mortality information and rates were also determined using data through December 31, 2014.
We also evaluated the proportion of individuals who had urine studies performed, particularly among those who had confirmed CKD by repeated creatinine measurement. Urine studies included urine dipstick, urine microscopy analysis, 24-hour urine protein quantitation, spot urine protein-to-creatinine ratio, and/or spot urine albumin-to-creatinine ratio. Last, we used the CKD-EPI equation to calculate eGFR and then stratified the study population using the new results. We compared the proportion of individuals who would have met the inclusion criteria for the creatinine safety program by the CKD-EPI formula vs the MDRD equation (which was originally used). The ESRD incidence and receiver operating characteristics curve were computed by the different equations for eGFR calculation. Also evaluated were multivariable Cox proportional hazards models examining ESRD or all-cause mortality outcomes adjusted for the confounding effects of baseline eGFR, age at first eGFR measurement, sex, race/ethnicity, Charlson Comorbidity Index, presence of proteinuria, and a history of stroke.
For descriptive statistics, continuous variables were reported as mean with standard deviation, median and interquartile range, and categorical variables were reported as the number and proportion at each level. Differences between groups for continuous variables or tests of association for categorical variables were made using t-test or nonparametric Wilcoxon rank sum tests and c2 or Fisher exact test, respectively, and as appropriate. Shapiro-Wilks test was used to determine normality for parametricity. All hypothesis tests conducted were 2-sided and considered significant at the 5% Type I error rate. All analyses and data management were conducted using SAS Enterprise Guide Version 5.1 (SAS Institute Inc, Cary, NC).
A total of 12,394 individuals were captured into the creatinine safety program in the period February 1, 2010, through March 1, 2014 (Table 1). On the basis of the initial abnormal creatinine measurement, 86.7% of the study population had an eGFR between 45 and 59 mL/min/m2. Among the 6980 individuals who eventually had a repeated creatinine measurement, 53.3% were found to have CKD (eGFR < 60 mL/min/m2).
End-stage Renal Disease and Mortality Outcomes
In the follow-up period up to December 31, 2015, a total of 83 individuals reached ESRD (56 among those with repeated creatinine measurements and 27 among those who did not follow-up with a repeated measurement; see Table 1). The mean follow-up was 4.2 years. The rate of ESRD by different categories of initial eGFR shows that most patients who reached ESRD came from the lower eGFR groups (< 45 mL/min/m2). The age- and sex-adjusted incidence of ESRD for the creatinine safety cohort during the study period was 94.7 per 100,000 person-years. Among the KPSC general population during the same period, the age- and sex-adjusted ESRD incidence was 35.4 per 100,000 person-years. Overall, the age- and sex-adjusted relative risk of ESRD was 2.68 times higher for the safety program cohort compared with the KPSC population (p < 0.001, Table 2, Figure 1). Among the study cohort, 301 patients died (159 who repeated and 142 patients who did not repeat their creatinine measurement).
Among the safety program cohort, multivariable adjusted Cox proportional hazards models for ESRD demonstrated that higher eGFR (hazard ratio [HR] = 0.57, 95% confidence interval [CI] = 0.53-0.62 for every 5-unit increase), male sex (HR = 0.58, 95% CI = 0.36-0.93), and older age at first eGFR (HR = 0.80, 95% CI = 0.74-0.87 for every 5-year increment) were associated with a reduced hazard for ESRD. Proteinuria was a substantial risk factor for ESRD in these models (HR = 5.37, 95% CI = 3.35-8.62) in addition to higher Charlson Comorbidity Index scores. Similar associations were seen for all-cause mortality, but higher age at first eGFR and male sex increased the hazard for death, demonstrating a competing risk between death and ESRD (Table 3).
Urine Studies Performed
Among the study cohort, 1602 individuals (12.9%) had any urine study performed (Table 4) within 180 days of their creatinine measurement. Among the individuals who went for a repeated creatinine measurement, 23.0% had a urine study performed. For those with confirmed CKD (second eGFR < 60 mL/min/m2), the proportion with a urine test of any kind within 180 days of the second eGFR was 31.8%. Among those with confirmed CKD and a repeated eGFR below 45 mL/min/m2, only 44.1% had a urine study of any kind performed. In total, 1619 of the 6980 with a repeated measurement had any urine test of any kind within 180 days of that measurement (Table 5).
Comparison of Equations
The redistribution of the study cohort based on the initial eGFR calculated using the CKD-EPI equation demonstrated that 4732 individuals (43.5%) would have had a calculated eGFR of 60 mL/min/m2 and higher. Thus, these individuals would not have been captured into the safety program if the CKD-EPI instead of MDRD equation had been used (Table 6). Only 5 of these individuals reached ESRD during our observation period. Thus, the CKD-EPI equation captured 56.5% of the original MDRD-based cohort but still included almost all patients who reached ESRD. With confirmed CKD based on the second eGFR as the predictive outcome, 67.8% of individuals captured by CKD-EPI would have had confirmed CKD compared with 52.3% using MDRD. Furthermore, using a logistic regression model to estimate the area under the receiver operating characteristics curve demonstrated that using the CKD-EPI equation for eGFR has higher validity compared with the MDRD equation (area under the curve = 0.943 vs 0.931, p = 0.025).
We sought to evaluate the impact of a creatinine safety program within the clinical care environment and infrastructure of an integrated health system (Kaiser Permanente). Our study evaluated ESRD outcomes on more than 12,000 individuals with single abnormal creatinine measurements who were captured into the KPSC creatinine safety program. We were able to demonstrate higher ESRD incidence rates among those who were captured into the safety program and still higher among those who had a repeated creatinine measurement. Black race, female sex, and proteinuria were associated with higher risk of ESRD in the safety program cohort. These findings support our assertion that the safety net is useful in identifying and outreaching to a high-risk population. We also found a potential care gap in that most of those who had confirmation of their CKD did not receive a urine study within 6 months. Last, our study findings demonstrated that using the CKD-EPI instead of the MDRD equation would have resulted in substantially fewer people being captured into the safety program while capturing almost all of those who progressed to ESRD.
Determining the added value of this creatinine safety program would validate the current efforts and help pave the way to improve the program moving forward. Most of our safety program population had early-stage CKD; 87% had eGFR in the range of 45 to 59 mL/min/m2. This population is not considered at high risk of ESRD or mortality as are later stages of CKD.14,24 Our findings demonstrate that a high-risk population for CKD was indeed identified while we were able to track pertinent practice patterns and CKD-related outcomes. The main assumption is that the program will lead to interventions that result in better overall CKD and CKD-related care. Early CKD education and management have been shown to result in improved pre-ESRD and post-ESRD outcomes,25 and this KPSC safety program was implemented to prevent delays in the diagnosis and treatment of CKD.
The method of screening and diagnosing CKD is an area of uncertainty. Measurement of eGFR is one way of assessing kidney function, but it is usually estimated with equations. KPSC laboratories report the eGFR using the MDRD calculation. However, the MDRD equation has systemic biases in that it was derived from a population already with a diagnosis of CKD.6,19 It has been described that up to 29% of healthy people had their kidney function underestimated on the basis of the MDRD equation.6 The CKD-EPI equation derived from people with and without CKD has been demonstrated to be superior to the MDRD equation, especially in the higher ranges of eGFR.7 This is particularly relevant to our current study cohort in which most patients were initially identified with marginally low eGFR. Furthermore, the CKD-EPI is a better prognosticator of ESRD and mortality outcomes.8,26 Similar to the findings of our study, the global estimate of CKD was lowered by 24% when CKD-EPI was used instead of the MDRD equation.8 Although it would identify some patients as having CKD that MDRD would not, the net persons identified with CKD would be less using the CKD-EPI equation.27 Given our findings, there may be an opportunity to refine the creatinine safety program by using a different eGFR equation. Using the MDRD equation may put the population at risk of overdiagnosis, potentially unnecessarily alarming patients. In addition, it may unnecessarily divert cost and resources from other clinical priorities. Within KPSC, had the CKD-EPI equation been used, 44% fewer individuals would have been contacted by the safety program.
The overall low rate of urine studies among patients with confirmed CKD in our study cohort appears alarming. Among patients with confirmed CKD in our study, only one-third had a urine study performed within six months. However, individuals with proteinuria had five times greater risk of progressing to ESRD compared with those without proteinuria. Markers of kidney damage such as proteinuria, hematuria, and anatomic abnormalities help define and prognosticate outcomes in CKD.12,28 Specifically, urine protein studies are recommended to use as a marker to manage CKD.29 Proteinuria has been associated with worsened cardiovascular outcomes, ESRD, and mortality.9-11 Although screening for CKD may be controversial, urine testing among patients with CKD is advocated by organizations such as the Kidney Disease Improving Global Outcomes, National Kidney Foundation, and American Diabetes Association.1,12
Chronic kidney disease is not always identified in an optimal and timely manner. Overall, there is low patient awareness and low clinician identification and documentation of CKD.16,17,30 Establishing a diagnosis of CKD is difficult because of a multitude of systemic, physician, and patient-oriented barriers.31,32 The chronicity needed for diagnosis, the lack of symptoms for many of the patients, and the large volume of laboratory results that flow through the routine clinical practice environment all contribute to a potential care gap leading to missed CKD diagnoses.
Although it appears that the creatinine safety program captures a high-risk population, patients who have severe or more urgent CKD were likely already captured and managed by the KPSC health system. One example is that members who have higher Charlson Comorbidity Index scores are less likely to be captured into the creatinine safety net (unpublished internal data, 2016). Another example is that when the KPSC SureNet program was started in February 2010, we dated our search to laboratory results from January 1997. However, only 13.5% of the safety program cohort was captured in the 13-year period from 1997 through 2009 compared with the remainder that were identified from 2010 through 2014.18 Already, KPSC has certain infrastructures in place to care for patients with a diagnosis of chronic conditions.33 Patients who have more obvious manifestations of CKD will more likely utilize the access to care that is readily available for all members. The Complete Care model at KPSC is inclusive of many of the tools and personnel needed to adequately ensure the success of safety nets such as the Creatinine SureNet.20 It takes advantage of the electronic health records and the integrated health system. It also has a model of being proactive toward patient care at all encounters and levels of care. One example is that the electronic charts have best-practice alerts that populate the screen for clinicians during patient encounters. Clinicians also have a proactive care screen section accessible during visits. The patients can use tools, such as the Internet portal (www.kp.org), to help awareness, communication, and follow-up. These online personal action tools have enabled faster closure of care gaps in KPSC.34
Future Direction of Creatinine Safety Research
Our research is currently funded (Agency for Healthcare Research and Quality, R01HS024437, principal investigator: KND), and we are studying more detailed aspects of care related to the creatinine safety program. Specifically, what are the contributors and barriers from a physician perspective that lead to care gaps and subsequent capture into the safety program? We are in the early stages of studying and identifying systemic, patient, and provider-related factors. Using our current and future findings, we hope to develop and implement preventive strategies to minimize the care gap. We need to continuously reflect on ways to “best cast the safety net” for the betterment of patient care.
Our study found a higher incidence of ESRD among individuals captured into the KPSC creatinine safety program. We found that the CKD-EPI instead of the MDRD equation would have identified 44% fewer individuals for the safety net while capturing almost all patients whose CKD progressed to ESRD. Among patients with confirmed CKD, 68% of patients did not receive urine testing in a timely manner. Although the creatinine safety program has an important role and place in an integrated health system such as Kaiser Permanente, our findings also suggest opportunities to improve CKD care and screening by refining this program.
This study was funded and supported by the Kaiser Permanente Southern California (KPSC) Clinician Investigator Award (JJS). The authors would like to thank David Selevan and Royann Timmins for their important work in the KPSC Creatinine SureNet program. The authors would also like to thank Mandy Cheung and Noel Pascual of the Renal Business Group for their assistance in information gathering and analysis pertaining to the chronic kidney disease and end-stage renal disease populations at KPSC. Finally, the authors wish to thank Stephanie D Goldman, MPH, Senior Consultant, Evidence-Based Medicine Services, Southern California Permanente Medical Group; for her assistance and evidence review comparing Chronic Kidney Disease Epidemiology Collaborarion and Modification Diet in Renal Diseases equations.
Kathleen Louden, ELS, of Louden Health communications provided editorial assistance.
How to Cite this Article
Sim JJ, Batech M, Danforth KN, Rutkowski MP, Jacobsen SJ, Kanter MH. End-stage renal disease outcomes among the Kaiser Permanente Southern California creatinine safety program (Creatinine SureNet): Opportunities to reflect and improve. Perm J 2017;21:16-143. DOI: https://doi.org/10.7812/TPP/16-143.
1. American Diabetes Association. Executive summary: Standards of medical care in diabetes—2011. Diabetes Care 2011 Jan;34 Suppl 1:S4-10. DOI: https://doi.org/10.2337/dc11-s004.