The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients


Pascal R Fuchshuber, MD, PhD, FACS; William Greif, MD, FACS; Chantal R Tidwell, RN; Michael S Klemm, MD; Cheryl Frydel, RRT; Abdul Wali, MD, FACP; Efren Rosas, MD; Molly P Clopp, RN, MS, MBATM

Winter 2012 - Volume 16 Number 1


The National Surgical Quality Improvement Program (NSQIP) of the American College of Surgeons provides risk-adjusted surgical outcome measures for participating hospitals that can be used for performance improvement of surgical mortality and morbidity. A surgical clinical nurse reviewer collects 135 clinical variables including preoperative risk factors, intraoperative variables, and 30-day postoperative mortality and morbidity outcomes for patients undergoing major surgical procedures. A report on mortality and complications is prepared twice a year.

This article summarizes briefly the history of NSQIP and how its report on surgical outcomes can be used for performance improvement within a hospital system. In particular, it describes how to drive performance improvement with NSQIP data using the example of postoperative respiratory complications—a major factor of postoperative mortality. In addition, this article explains the benefit of a collaborative of several participating NSQIP hospitals and describes how to develop a "playbook" on the basis of an outcome improvement project.


The National Surgical Quality Improvement Program (NSQIP) is a surgical outcomes database of the American College of Surgeons (ACS) designed to measure risk adjusted outcomes of surgical interventions so as to compare results between hospitals.1 This is achieved with a validated risk adjustment using a logistic regression model. This risk adjustment allows unbiased comparison of results between hospitals of different sizes serving different patient populations.2,3


NSQIP was born in the mid-1980s from a US government mandate4 to improve surgical outcomes among 133 Veterans Administration (VA) hospitals because of a high-observed rate of mortality and surgical complications. This provision was hampered by the absence of a risk-adjusted database and of a reliable national average for surgical outcomes. Therefore the legislative act provided for the development of a risk-adjustment model that would take into account the severity of a patient's illness and thus enable "apple-to-apple" comparisons of results from participating hospitals.

Following the National VA Surgical Risk Study conducted by the Department of Veterans Affairs between October 1, 1991, and December 31, 1993, parameters of risk adjustment were established5 and the VA-NSQIP was created in 1994 allowing risk-adjusted comparison of the 133 VA hospitals.6 This constituted the birth of NSQIP. By mandate all VA hospitals were required to participate. The program has proved so successful—with NSQIP participation, mortality and morbidity in the VA system was reduced by 27% and 45%, respectively5—that the ACS adopted NSQIP and tested it in a small number of academic medical centers and private hospitals. The results in this initial cohort of private hospitals validated the VA results and were very well received. In 2004, NSQIP of the ACS (ACS NSQIP) was initiated at the national level (Figure 1). Currently there are over 350 participating hospitals in the US and the Middle East.

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients


How It Works

NSQIP is based on 135 variables collected preoperatively and up to 30 days postoperatively. The variables are collected from the following categories: 1) demographics, 2) surgical profile, 3) preoperative, 4) intraoperative, and 5) postoperative data. Each hospital submits an average of 1600 major operations per year into the NSQIP database. Nine categories of complications are reported: 1) overall mortality; 2) overall complications; 3) cardiac complications; 4) postoperative pneumonia; 5) intubations required within 48 hours postsurgery (>48-hour intubations); 6) unplanned intubations; 7) pulmonary embolism and venous thrombosis; 8) renal dysfunction; and 9) surgical-site infections including superficial, fascia, and deep infections.

The collection of variables is performed by a Surgical Clinical Reviewer trained by NSQIP in the analysis of medical records. It is noteworthy that in hospital systems with established electronic medical records, such as Kaiser Foundation Hospitals, reliability of the data records is consistently very high. The Surgical Clinical Reviewer is audited yearly to ensure reliability of abstraction to at least 95%. The strength of this system lies in the strict definition of complications and validation of risk adjustment. For example, the definition of postoperative pneumonia is predetermined for all participants. Appropriate adherence to these definitions is monitored through regular audits carried out by the central NSQIP office.

The variables are subjected to logistic regression to determine their ability to predict the risk of complications.7 Table 1 shows the ranking by importance of the first nine variables. The consistency of the three primary factors (albumin, presence of cancer, American Society of Anesthesiologists [ASA] score) demonstrated a robust system. The risk adjustment allows precise comparison of hospitals. Figure 2 represents the importance of risk adjustment according to two hospital reviews of surgical mortality. This ranking is reversed when the risk adjustment is introduced, ie, the ratio of observed results and expected results. The expected results are based on NSQIP data using the risk-adjustment variables described above. Without interference from the participating hospitals, Outcomes Sciences, Inc (Cambridge, MA), an independent data management company, performs the calculation of the risk-adjusted outcomes using the patient variables submitted by each hospital.

Display of National Surgical Quality Improvement Program Results

NSQIP results are presented through a "caterpillar" chart depicting observed vs expected ratio for each participating hospital and for each category of complication. Each line corresponds to the result of one particular hospital with the confidence interval (CI) included. The most successful hospitals are to the left of the graph in Figure 3, the worst performers to the right. When the results of a particular hospital are significantly better than the average, the CI line is entirely below the mean (horizontal black line); when the results of a particular hospital are significantly worse than the average, the CI line is entirely above the mean. This "Risk-Adjusted Report" is generated every six months for each category of complication for all participating hospitals. More recently this information has been depicted in bar graphs, which allows for a more concise visual demonstration of these findings (Figure 4).

A graph is generated for each hospital showing the risk-adjusted results for all of the categories of complications (mortality, overall morbidity, cardiac complications, pneumonia, unplanned intubations, >48-hour intubations, deep venous thrombosis and pulmonary emobolism [DVT/PE], renal failure, urinary tract infections [UTIs], and surgical site infections) with hospital-specific ranking indicated by a large arrow (Figure 5). This allows each hospital to evaluate its own performance as compared with other participating hospitals.8 Each category graph includes a small insert that shows the performance of the specific hospital throughout the period of participation. Each individual hospital's performance can thus be followed over time.

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients


Intubation >48 Hours

After receiving the first 12 months of risk-adjusted data from NSQIP, the Kaiser Permanente Northern California (KPNC) Walnut Creek Medical Center NSQIP group chose its first performance-improvement project on the basis of outcome information on >48-hour intubations. This was an obvious choice because Walnut Creek Medical Center ranked among the significantly worst performers for this complication. The observed incidence was almost twice that of the expected rate after risk adjustment (Figure 6). We benefited from the simplicity of the processes involved to improve this parameter compared with other complications that required more complex processes, such as pneumonia or surgical-site infection. When choosing a first performance-improvement project as a new participant of NSQIP, it is important to consider the complexity of the processes involved and to choose one in which the parts involved are relatively easy to decipher. For example, UTI of >48-hour intubations is a complication with relatively low levels of complexity and a low number of involved processes compared with pneumonia or mortality.

Principal steps of the performance improvement project >48-hours intubation
Create A Team
•    Surgeon, anesthesiologist, intensivist, Intensive Care Unit (ICU) Nurse Manager, Respiratory Therapy Department Manager, frontline Respiratory Therapist (RT), Quality Department representative, Post-Anesthesia Care Unit (PACU) Representative, Nurse Manager
Identify Issues
•    PACU does not communicate
with RT
•    Limited availability of RT in ICU
•    Lack of involvement of RT
Change Practice
•    Educate RT on weaning within
48 hours
•    Audit patient charts to assess weaning criteria and do multiple attempts of weaning
•    Document why patient failed <48-hour weaning
•    Add second shift for RT in ICU
•    Specific weaning order set
•    RT to start setting ventilator in PACU and to provide weaning parameters
•    Anesthesia, operating room staff and RT communicate on expected extubation time
•    Night shift nurse decrease sedation as updated in the sedation/vacation policy
•    Night shift RT begin weaning at 5:00 am for early morning extubation

In response to our risk-adjusted results for >48-hour intubations, we implemented an improvement project with 3 main steps (see Sidebar: Principal steps of the performance-improvement project: >48-hours intubation): 1) creation of a working group comprising all stakeholders; 2) identification of all operational problems; 3) development of processes to change practice (small test of change); and 4) verification of results using the NSQIP database, both the risk-adjusted results (Figure 6) and the raw data results (Figure 7). It is important to mention here that both risk-adjusted data and raw data from the NSQIP database are helpful in guiding a performance-improvement project. Whereas risk-adjusted data gives the most "objective" rank of performance and allows comparison to group performance (Figure 6), raw data (Figure 7) allow a rapid assessment of progress made on a short-term basis with a "small test of change" improvement project. Critical adjustments and changes can be made on the basis of the results of raw (run chart) data.

The main results of this project were the identification of major communication shortcomings among the clinicians caring for patients in the immediate postoperative period. The lack of efficient communication between the treatment team and the respiratory therapy team, and the relative absence of respiratory care during postoperative recovery of patients because of staffing issues were discussed and presented to the individual department and the hospital leadership. It is important to note that all problems identified were related to systems and not individual performance of a surgeon, nurse, or respiratory therapist (RT). This is the case for almost all problems identified by NSQIP.

Performance improved within one year after the implementation of a number of small tests of change. The changes implemented included: identifying patients at risk, expanding respiratory care coverage, and avoiding oversedation of patients scheduled to be weaned. We have moved from right to left on the caterpillar graph with an important reduction in the absolute and risk-adjusted incidence of prolonged postoperative intubations (Figure 6). Our observed versus expected quotient is now as expected on risk-adjusted analysis. Interestingly, the mortality associated with this complication was reduced as well (Figure 7).

Postoperative Pneumonia

The success of this project encouraged us to expand and to address the reduction of postoperative pneumonia, a more complex complication with less well-defined operational issues. Again, we implemented an improvement program including pre-, intra- and postoperative interventions with education and enforcement both in the surgery clinic, Post-Aesthesia Care Unit and on the wards (see Sidebar: Principal steps of the performance-improvement project: postoperative pneumonias). We included in our program for the prevention of postoperative pneumonia the processes listed in the Sidebar: The postoperative-pneumonia prevention bundle, including description of all perioperative interventions. This bundle was initially implemented in high-risk patients undergoing major upper abdominal surgery but has since been expanded to include any major abdominal surgery requiring hospital admission beyond one day. It is apparent that this project exceeds the complexity of the previous project by several magnitudes. The difficulty resides in the details of implementation and ensuring close to 100% adherence to the processes in clinical practice. This requires a coordinated approach by the nursing leadership and strong motivation and buy-in of the involved caregivers: surgeons, anaesthesiologists, nurses, nursing assistants, RTs, and medical assistants.

The incidence of postoperative pneumonia declined considerably and currently places our hospital among the best performers (Figure 8). More compelling is the raw data on the incidence of postoperative pneumonia. The absolute number of patients with pneumonia following general surgery remained at zero from May 2010 until February 2011. This represents an average reduction of 2 to 3 postoperative pneumonias per month in general surgery patients (Figure 9). Since February 2011, we have seen a few pneumonias in our patient population, but the observed incidence places us well below the mean observed versus expected ratio in the entire NSQIP cohort. Clearly our next challenge is to sustain the achieved reduction of postoperative pneumonias in our patient population

National Surgical Quality Improvement Program Consortium

The role of a NSQIP consortium is to create a synergy between hospitals participating in a region.9,10 It has been shown that regional cooperation can reduce expense and surgical complications.11
In 2009, an Interregional NSQIP consortium with participation of all KP Regions was created assembling 18 NSQIP participating Kaiser Foundation Hospitals. The consortium serves as a platform to exchange and to share the results among all participating hospitals, to learn from the best and worst results, and to develop common improvement projects. A "playbook" is created from successful projects and distributed to all hospitals within the consortium. On the basis of shared information, monthly meetings, and occasional site visits, the consortium partners learn from each other and share best practices. In 2011, a Northern California NSQIP collaborative was formed as participation in NSQIP was spread to all 21 KPNC Medical Centers.

Create a Playbook

A playbook is a compilation of all elements necessary to implement a performance-improvement project after successful testing at one or two beta sites. For example, our KPNC collaborative developed a playbook on the basis of the successful implementation of a performance-improvement project focused on wound classification in the operating room. The goal of this project was the correct classification of wounds to assure greatest accuracy of the risk-adjusted reporting of postoperative wound infection into the NSQIP database. This is a good example of how to address one of the key issues with outcomes databases such as NSQIP: how to ensure validity and accuracy rate of the reported data. Our goal was an accuracy of at least 90% in postoperative documentation of wound class.

Principal steps of the performance-improvement project: postoperative pneumonias
Patient group
•    All elective gastrointestinal, colorectal, and thoracic surgical patients
Preoperative interventions
•    Incentive spirometer
•    Oral care: chlorhexidine rinse
Postoperative interventions
•    Oral care
•    Head of bed elevated
•    Incentive spirometer
•    Mobilization
•    Percentage who received a bundle in preoperative care
•    Percentage who received a bundle in postoperative care
•    Number of pneumonias and pneumonia rate by NSQIP
•    Number of deaths and mortality by NSQIP

In this example, the process of Plan-Do-Study-Act (PDSA) was used. An improvement plan was set in motion with successive re-evaluations and adjustments. More rapid cycles of PDSA followed to improve the education process and verification of the classification of wounds (Figure 10). Figure 11 shows the gradual achievement of at least 90% accuracy in wound classification. As part of the playbook, posters with the definition of wound classification (per the ACS)12 are distributed in all operating rooms. The classification of wounds is now part of the formal debriefing after surgery. The educational materials, posters, and debriefing checklists are now used by all operating rooms in the KPNC collaborative.

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients

The postoperative-pneumonia prevention bundle, including description of all perioperative interventions
Preoperative clinic
•    Patient gets pneumonia-prevention education (spirometry) by medical assistant
Preoperative care
•    Patient is given chlorhexidine oral care (ordered by the surgeon on preoperative orderset)
Postoperative care
•    Clinician orders chlorhexidine, head-of-bed elevation, and incentive spirometry
•    Nursing floor leadership ensures nursing staff is aware that:
1. Head-of-bed must be elevated at least 30 degrees at all times, unless medically contraindicated
2. Incentive spirometer means 10 workouts per day, used correctly (ie, inspire up to 500ml 10 times per workout). If the patient is unable to meet the goal, the nurse must call the surgeon (consider albuterol nebulizer) and respiratory therapist to evaluate for chest physiotherapy
3.    Ambulation goals: day of surgery: dangle legs; postoperative day 1: stand and sit in a chair 3 times; postoperative day 2 and thereafter: ambulate 100 feet 3 times per day. If the patient is unable to meet any of these goals, the nurse must call the surgeon (treat pain and hypotensive issues) and consider a physiotherapy consult
4.    Aspiration precautions: flush nasogastric tube and confirm sump is functioning properly every 4 hours. If it is not functioning, call a physician for suggestions and/or evaluation
5.    When the patient begins to eat (likely not until postoperative day 3 or 4), the patient is then out of bed and sitting in a chair for all meals

The Power of the National Surgical Quality Improvement Program—Achieving A Zero Pneumonia Rate in General Surgery Patients



NSQIP is central to our efforts to achieve high-quality and effective surgical care for our patients. It has been shown that by using processes similar to the ones described in this article, participating institutions can reduce postoperative mortality and complications by an average of 27% or 45%, respectively. Each participating NSQIP hospital has been shown to eliminate an average of 250 postoperative complications and 12 to 27 surgical deaths per year.8 The economic implications are impressive. The Walnut Creek Medical Center has saved more than 1 million USD per year since the implementation of the NSQIP. This far exceeds the cost of the program of (100,000 to 150,000 USD per year): 35,000 USD for the processing of data and 50,000 to 100,000 USD to hire the NSQIP Surgical Nurse Reviewer (US wages) for data collection.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

1.    National Surgical Quality Improvement Program [home page on the Internet]. Chicago, IL: American College of Surgeons; 2005 [cited 2012 Jan 17]. Available from:
2.    Dimick JB, Osborne NH, Hall BL, Ko CY, Birkmeyer JD. Risk adjustment for comparing hospital quality with surgery: how many variables are needed? J Am Coll Surg 2010 Apr;210(4):503-8.
3.    Ingraham AM, Richards KE, Hall BL, Ko CY. Quality improvement in surgery: the American College of Surgeons National Surgical Quality Improvement Program approach. Adv Surg 2010;(44):251-67.
4.    Veterans' Administration Health-Care Amendments of 1985, Pub L No 99-166, 99 Stat 941. (Dec 3, 1985); Title II: Health-Care Administration, Sec. 201–4.
5.    Khuri SF, Daley J, Henderson WG. The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs. Arch Surg 2002 Jan;137(1):20-7.
6.    Khuri SF, Daley J, Henderson W. The Department of Veterans Affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg 1998 Oct;228(4):491-507.
7.    Cohen ME, Dimick JB, Bilimoria KY, Ko CY, Richards K, Hall BL Risk adjustment in the American College of Surgeons National Surgical Quality Improvement Program: a comparison of logistic versus hierarchical modeling. J Am Coll Surg 2009 Dec;209(6):687-93.
8.    Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 2009 Sep; 250(3):363-76.
9.    Campbell DA Jr, Kubus JJ, Henke PK, Hutton M, Englesbe MJ. The Michigan Surgical Quality Collaborative: a legacy of Shukri Khuri. Am J Surg 2009 Nov;198(5 Suppl):S49-55
10.    Campbell DA Jr, Englesbe MJ, Kubus JJ. Accelerating the pace of surgical quality improvement: the power of hospital collaboration. Arch Surg 2010 Oct;145(10):985-91.
11.    Share DA, Campbell DA, Birkmeyer N, et al. How a regional collaborative of hospitals and physicians in Michigan cut costs and improved the quality of care. Health Aff (Millwood) 2011 Apr;30(4):636-45.
12.    Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for prevention of surgical site infection, 1999. Hospital Infection Control Practices Advisory Committee. Infect Control Hosp Epidemiol 1999 Apr;20(4):250-80.



The Permanente Journal is celebrating it's 20th anniversary year. We look forward to continuing to bring you more high-quality content during the next 20 years.


Sponsored by the National Permanente Medical Groups, The Permanente Press publishes The Permanente Journal and books related to Kaiser Permanente and health care.


25,000 print readers per quarter, 7,628 eTOC readers, and in 2016, 1.4 million page views on TPJ articles in PubMed from a broad international readership.


Articles, editorials, letters to the editor, and other material represent the opinion of the authors. Send your comments to

Copyright 2017 The Permanente Journal - Kaiser Permanente. All Rights Reserved.