Predicting Risk of Death in General Surgery Patients on the Basis of Preoperative Variables Using American College of Surgeons National Surgical Quality Improvement Program Data![]() Sachin Vaid, MD; Ted Bell, MS; Rod Grim, MA; Vanita Ahuja, MD Perm J 2012 Fall;16(4):10-17https://doi.org/10.7812/TPP/12-019AbstractObjectives: To use the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) database to develop an accurate and clinically meaningful preoperative mortality predictor (PMP) for general surgery on the basis of objective information easily obtainable at the patient's bedside and to compare it with the preexisting NSQIP mortality predictor (NMP). IntroductionAccurate estimation of the risk of death can help patients and their physicians to make decisions and to manage expectations; however, surgeons lack an easily accessible preoperative bedside evaluation system to calculate mortality. Various scoring systems to assess perioperative mortality have been reported in the literature, including the Physiologic and Operative Severity Score for the enUmeration [sic] of Mortality and Morbidity,1 Surgical Risk Score,2 Biochemistry and Haematology Outcome Models,2 Acute Physiology and Chronic Health Evaluation,3,4 Cleveland Clinic Foundation Colorectal Cancer Model,5 and the French Association of Surgery's colorectal scale.6,7 However, these are not easily calculated at the bedside. None of these scoring systems has the ability to predict accurately death or survival solely on the basis of preoperative variables, nor do they account for interhospital variability in outcomes. The American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) database has been described by the Institute of Medicine as the best in the nation for measuring and for reporting surgical quality and outcomes. It is based on three important quality measurement principles: 1) data are collected and independently entered by specially trained nurse reviewers, thus eliminating the possible bias inherent in traditional surgical databases; 2) all cases are followed up for 30 days, even if the patient is discharged from the hospital, thus increasing the accuracy and thoroughness of reporting; and 3) cases are categorized by current procedural terminology codes, which standardizes analysis and reporting across institutions. NSQIP has developed models that use these data to predict mortality and morbidity across broad categories, such as general and vascular surgery.8 ACS NSQIP assesses surgical quality at more than 200 hospitals in the US by collecting thorough data on preoperative risk factors and postoperative morbidity and mortality.8 Comprehensive computerized procedures and on-site auditing ensure data integrity. NSQIP also has a reliable postdischarge mortality predictor. The NSQIP mortality predictor (NMP)9-11 is calculated from 35 variables related to the patient's physiologic status, demographic data, medical history, laboratory values, and American Society of Anesthesiologist (ASA) scores. It was designed with stepwise logistic regression. The probabilities from the regression analyses yield a probability (0 to 1) that a patient will experience a mortal event on the basis of the preexisting, or preoperative, conditions. These numbers can be converted to percentages. Therefore, a 0.59 probability can be expressed as a 59% chance of death. ObjectivesThe purpose of this study was to use the ACS NSQIP database to develop an accurate, reliable, and clinically meaningful preoperative general surgery model to predict death or survival on the basis of information obtainable at the patient's bedside. NMP, the ACS NSQIP risk predictor, is available in the semiannual report provided to the hospital; however, the up-to-date NMP is not available to the surgeon at the time of patient admission. The preoperative mortality predictor (PMP) we developed in this study is based entirely on objective preoperative variables and is designed to be useful during preoperative counseling and the informed consent process. MethodsOverviewA model to predict death or survival was developed and tested using data from the ACS NSQIP database for patients who underwent common surgical procedures from January 1, 2005, through December 31, 2009. The project was reviewed and approved by the WellSpan Health Medical/Surgical institutional review board. The study comprised three steps. Step 1 was to create the PMP using data from 2005 through 2008. Step 2 was to apply the PMP to the 2009 data for verification. Step 3 was to compare the PMP with the NMP to determine the accuracy of the new tool. Patient Selection and Data CollectionFor the present study, two datasets (2005 to 2008, 2009) were created from the ACS NSQIP database. The goal was to create the PMP model using the 2005 to 2008 dataset and then to apply it to the 2009 dataset to test its accuracy. We applied the same inclusion criteria to all data. All eligible patients were included. Inclusion criteria were age 18 years or older, and 1 of 7 common surgeries: open pancreas surgery or open/laparoscopic colorectal, hernia (ventral, umbilical, or inguinal), or gallbladder surgery (Table 1). These 7 categories of surgery were chosen to represent 2 aspects of general surgery: the variety of cases,12 and their operative complexity. Regarding variety, the procedures we selected comprise a mixture of elective and emergent surgeries, such as perforation and obstruction. They also address the concern raised by Aust et al, that operative complexity must be evaluated to accurately estimate operative mortality.13 We included all variables available in the ACS NSQIP database. Some variables were redefined to facilitate analysis. For example, patients were categorized as surviving or dead on the basis of year of death. To assess the effects of comorbidities on mortality, we created summary variables: pulmonary (including ventilator, history of chronic obstructive pulmonary disease, and current pneumonia), liver (including ascites and esophageal varices), renal (including dialysis and renal failure), neurologic (including history of transient eschemic attack and cardiovascular event with or without neurologic deficit), and cardiac (including dyspnea and history of congestive heart failure, myocardial infarction, angina, peripheral artery disease, and hypertension requiring medication) morbidity. Poor functional health status was defined as total assistance required for all daily activities (see Sidebar: National Surgical Quality Improvement Program definition: poor functional health status).
In an exploratory analysis of the entire dataset, only 4 (blood urea nitrogen, white blood cell count, serum glutamic-oxaloacetic transaminase, and partial thromboplastin time) of 13 (30.8%) laboratory variables were statistically significant predictors of death, with an odds ratio (OR) barely greater than 1. The statistically significant OR for albumin was less than 1 (0.350), indicating that death is less likely to occur on the basis of albumin levels. However, for 8 of the 13 (61.5%) laboratory variables, 19% (hematocrit) to 75% (albumin) of data were missing. Instead, we used a variable that has been shown to be an independent predictor of death and that is easy to determine from medical history: loss of >10% body weight in the last 6 months.14 Weight loss is a predictor of death after intra-abdominal surgery and lung surgery.15,16 The predictive ability of ASA score has been debated since its inception because of its subjective descriptions.17-19 In the exploratory phase of the present study, ASA strongly predicted operative death, and its contribution to the mortality score far outweighed other variables. We excluded ASA from our analysis because we wanted to investigate objective factors on the basis of the history taken by and the physical examination performed by the surgeon. Recently, concerns have been raised that ASA class can be manipulated so it appears that the hospital is caring for sicker patients. ASA may lack precision and poor inter-rater consistency, which can lead to inaccuracy. There is significant interdependence between ASA and other preoperative NSQIP variables that can be used to calculate risk.20,21 Statistical Analysis and Development of the ScoreAll analyses were performed using SPSS 19 (IBM, Armonk, New York). To create the PMP model with the 2005 to 2008 data, we selected a set of preoperative variables on the basis of published studies.22,23 We conducted a χ2 analysis to determine which preoperative variables were significantly associated with death. We analyzed those significant variables with logistic regression, which identified a number of variables with high ORs (β > 1.3, p < 0.001). To construct the scoring system, we performed a frequency analysis of the variables (for deceased patients only) that were significant on logistic regression (p > 0.05). The purpose of the frequency analysis was to determine the frequency of these characteristics among patients who died in hospitals nationwide. The ORs calculated with logistic regression were then weighted according to the frequencies of the corresponding characteristics to arrive at a score for each variable. A 1-point increase in score would indicate a predictable increase in overall mortality. Therefore, each variable's score was based on 2 factors: 1) its OR and 2) the frequency of the corresponding characteristic in patients who died in hospitals nationwide. We compared the PMP to the NMP using Spearman correlation, a nonparametric test to determine the strength of association between two variables, with scores ranging from -1 (no correlation) through 1 (perfect correlation). Typically, results are presented as percentages; eg, a 0.85 Spearman correlation coefficient means that 85% of the variation in one variable is related to variation in the other. To test the accuracy of the PMP and NMP, three logistic regressions (2005 to 2008 data, 2009 data, and combined data) were run to determine how well the PMP and NMP predicted death. Logistic regression uses both categorical and quantitative variables to predict a dichotomous variable. The classification number generated indicates how well a group of variables predict another variable. In this study, death was the dichotomous variable being predicted by the quantitative variables of the PMP and NMP. The PMP was further tested with a receiver operating characteristic curve. Receiver operating characteristic curves are used to test discrimination, or, in the case of this study, the ability of a model to distinguish between dead and surviving patients. This is tested using the area under the curve, which is expressed as a percentage and indicates how well the model discriminates between dead and surviving patients. ResultsThe analysis includes 202,741 (68%) patients from the 2005 to 2008 period and 94,060 (32%) patients from 2009. Demographic and clinical patient characteristics were compared by year (Table 2). A χ2 analysis of 2005 to 2008 patient characteristics by outcome (death/survival) showed that 18 variables are significantly associated with death (Table 3). Logistic regression showed that 15 of the 18 variables (including 3 age groups) were strong predictors of death or survival (Table 4). Some of these variables had a stronger association with death than others. For example, 98% of those who died were inpatients, whereas only 11% of those who died had disseminated cancer. The value of the logistic regression coefficient β was then weighted by the frequency of the characteristic to provide a score. Scores were assigned to each variable on the basis of their OR and frequency (Table 5). PMP scores could range from -1 through 30. Spearman correlation of NMP and PMP scores for the 2005 to 2008 data was 87.5%. Spearman correlation of NMP and PMP for the 2009 data was 85.5%. Spearman correlation of NMP and PMP for all (combined) data was 86.9% (linear relationship) (Figure 1). As NMP scores increase, so do PMP scores. Regression results for 2005 to 2008 data, 2009 data, and all (combined) data indicated that PMP score was statistically reliable (p < 0.05) in distinguishing between dead and surviving patients and correctly classified 98.7%, 98.6%, and 98.6% of cases, respectively. Receiver operating characteristic curve analysis found that the PMP was 93% accurate at predicting death (Figure 2). Differences occurred in outcomes among patients with different scores. When scores were grouped into 5-point intervals, mortality rate increased with increasing interval (Figure 3). For example, <1% of people in the 0-5 score range died, whereas 73% of those in the 20 to 30 range died. DiscussionOperative mortality can be used to compare quality of care in different health care settings. This information is sought by both the payer and the patient and is also valued by surgeons assessing their outcomes. PMP score accurately predicted death or survival after a common general surgery operation. This information could be used as a standardized indicator to compare observed and expected mortality rates at different hospitals and to assess their performance. Additionally, the PMP could be a useful audit tool for surgeons. Optimal patient care requires a successful interface between the surgeon, operating room staff, nursing staff, and ancillary staff. The success of this interaction reflects the performance of the hospital. The PMP could be used as a reliable measure of quality at the system level, replacing older tools that focused only on patient variables. In our analysis, inpatient status had the greatest contribution to PMP score. The practice of ambulatory surgery is increasing in the US, but inpatient surgery is still required for sicker patients and for more complex operations. Inpatient status exposes patients to hospital-acquired infections, which contributes indirectly to mortality. Five percent to 10% of patients admitted to acute care hospitals in the US will acquire a nosocomial infection. Health care-associated bloodstream infections are associated with attributable mortality that makes them the eighth-leading cause of death in the US.24 Our analysis reveals that advanced age independently predicted death or survival. The assigned score for age increased from 1 to 1.5 to 2.5 as patients moved between age groups, from 65-69 to 70-79 to ≥80 years. Several studies have confirmed old age as a risk factor for operative mortality. 25-31 Advanced age leads to progressive decline in physiologic reserve and ability to compensate for stress. Elderly patients have both more and more serious medical comorbidities, which may be related to delayed presentation, poor communication, or increased incidence of certain diseases, such as colorectal cancer and diverticulitis. Abdominal pathology in older patients often presents emergently.32,33 Elderly patients are a vulnerable group and need special attention and care. Health care professionals may have to alter practices to accommodate their needs, eg, by minimizing sleep interruptions, altering dosage and type of medicines, or having a geriatrician follow-up with the patient while in the hospital. Improving perioperative practices could be one avenue to lower mortality among geriatric patients. Approximately 403,000 cholecystectomies were performed in the US in 2007.34 The mortality rates associated with these common general surgery procedures are minimal but may escalate if cholecystectomy is performed in elderly patients in emergent settings, as shown by Dolan et al35 and Kurian et al.36 More than 33% of cholecystectomies and 50% of partial colectomies were being performed in patients older than age 65 years. In 2007, those aged at least 65 years accounted for just 13% of the US population but 37% of hospital discharges and 43% of days of care.34,a Poor functional status (PFS) and do-not-resuscitate (DNR) directives were independent predictors of mortality.37-40 Patients with PFS and DNR directives are generally sicker, have more comorbidities, are more vulnerable to neglect, and may present later in their disease process, which may result in higher mortality. The decision to operate on patients with PFS and DNR directives should be thoroughly reviewed with a detailed preoperative workup and optimization. A study in the United Kingdom also indicated that patients with DNR directives may require specialized care, and limited access to such care may contribute to higher mortality.41 PFS has been shown to be a strong predictor of postoperative cardiac42 and neurologic43 adverse events. Patients with renal failure who are on hemodialysis and patients with cirrhosis are known to have higher postoperative mortality rates than their age peers.44-47 The present study identified liver and renal failure as independent risk factors that predicted death. Medical comorbidities such as cardiac problems, pulmonary dysfunction, and bleeding disorders also independently predicted death. Preoperative optimization of these conditions may be an avenue for risk reduction. Sepsis emerged as an independent predictor of death. Patients presenting in sepsis are generally operated on emergently. The mortality rate associated with emergency laparotomies is greater than mortality for elective operations, which has also been reported by McIlroy et al48 and Pederson et al.49 The hospitalization rate of those with a principal diagnosis of septicemia or sepsis more than doubled from 2000 to 2008, increasing from 11.6 to 24.0 per 10,000. In-hospital deaths increased from 2% to 17% among patients hospitalized with a diagnosis of sepsis, compared to other diagnoses. For patients younger than age 65 years, 13% who were hospitalized for septicemia or sepsis died in the hospital, compared with 1% of those hospitalized for other conditions. For patients age 65 years and older, 20% of septicemia or sepsis hospitalizations ended in death, compared with 3% for other hospitalizations.24 Adhering to the Surviving Sepsis Campaign50 and similar guidelines could potentially decrease mortality. Steroid use in the preoperative phase was an independent predictor of death. Ismael et al, in their study using 2005 to 2008 NSQIP data, demonstrated that steroid use increased the rate of surgical site infections but also may have increased mortality almost fourfold.51 Presence of disseminated cancer also independently predicted death. A recent study by Kwok et al has shown that metastatic cancer is the only diagnostic variable that is a significant predictor of death.52 Surgical complexity was only minimally predictive, and only in very complex procedures, such as pancreatectomy. Aust et al proposed a similar hypothesis about surgical complexity.13 PMP score can be used as a quick bedside/office tool to determine a patient's risk of perioperative death. In contrast to the risk calculators developed by Khuri9-11 and Aust,13 the PMP does not require any laboratory data to predict mortality. The strength of the present study is that we incorporated a high volume of patients by including data from 2005 through 2008. This study was able to successfully validate the PMP with 2009 ACS NSQIP data. The limitation may be that the results cannot be generalized to institutions that do not participate in the ACS NSQIP. Significant differences between ACS NSQIP and non-ACS NSQIP hospitals seem unlikely given the diverse array of representation in the current ACS NSQIP. It would be interesting to see if this tool can be successfully used in health care settings outside the US. ConclusionThe PMP is an accurate, simple, effective, and clinically meaningful tool to calculate the risk of perioperative death using only preoperative variables. Risk can be easily calculated at the bedside without any laboratory values. The PMP would give physicians the ability to reliably report the risk of death for a wide range of patients undergoing common general surgery procedures, both elective and emergent. It would be helpful in providing accurate information about the risk of death and in obtaining informed consent. In addition to being a useful counseling tool, the PMP could also provide data on the performance of surgeons and hospitals. a Starting from 2002, the expression "days of care" was replaced by the expression "occupancy/billing days." This is an adaption to the terminology of the lump-sum pay system. In the field of the prevention or rehabilitation facilities the expression "days of care" further exists. The number of occupancy/billing days is the sum of the fully inpatient patients of each day of the reporting year at 12 pm (sum of midnight stock). The reported values of occupancy/billing days include the hourly cases. Days of intensive care treatment/intensive care monitoring are occupancy/billing days for patients who are treated in intensive care beds. Disclosure StatementThe author(s) have no conflicts of interest to disclose. AcknowledgmentLeslie E Parker, ELS, provided editorial assistance. References 1. Copeland GP, Jones D, Walters M. POSSUM: a scoring system for surgical audit. Br J Surg 1991 Mar;78(3):355-60. DOI: https://doi.org/10.1002/bjs.1800780327
|
ETOC
Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.
CIRCULATION
2 million page views of TPJ articles in PubMed from a broad international readership.
Indexing
Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, and CrossRef.