|
|
||||||||
Ann Thorac Surg 2007;84:1256-1262
© 2007 The Society of Thoracic Surgeons
a Department of Nephrology, Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Taipei, Taiwan
b Respiratory Care Department, Chang Gung Institute of Technology, Chia-Yi, Taiwan
Accepted for publication May 18, 2007.
* Address correspondence to Dr Chen, Division of Critical Care Nephrology, Department of Nephrology, Chang Gung Memorial Hospital, 199 Tung Hwa North Rd, Taipei, 105, Taiwan (Email: cyc2356{at}adm.cgmh.org.tw).
| Abstract |
|---|
|
|
|---|
Methods: We reviewed the medical records of 78 critically ill patients on ECMO support at the specialized intensive care unit in a tertiary care university hospital from March 2002 to October 2005. Demographic, clinical, and laboratory variables and five scoring systems were retrospectively gathered as predicators of survival on ECMO day 1.
Results: The overall mortality rate was 60.3%. The most common condition requiring ECMO was cardiogenic shock. Goodness-of-fit was good for APACHE IV but not the APACHE III model. The APACHE IV and APACHE III scoring systems displayed excellent areas under the receiver operating characteristic curve (0.922 ± 0.030 and 0.907 ± 0.038, respectively). Furthermore, APACHE IV correlated significantly with APACHE III scores in individual patients (r
2 = 0.902; p < 0.001). Finally, cumulative survival rates at 6-month follow-up after hospital discharge differed significantly (p < 0.001 for APACHE IV
49% versus APACHE IV >49%).
Conclusions: This study confirms the grave prognosis of critically ill patients receiving ECMO support. The APACHE IV proved to be a reproducible evaluation tool with excellent prognostic abilities in these patients.
| Introduction |
|---|
|
|
|---|
Previous reports reveal a poor prognosis for patients receiving ECMO [1, 2]. When treating critically ill patients, objective severity assessment is important in selecting therapeutic approach, as well as in comparing the benefits of various treatments, assessing new therapeutic procedures, comparing treatment success rates among medical centers, and explaining the patients condition to family members [6, 7].
The Acute Physiology and Chronic Health Evaluation (APACHE) II and III are physiologically based scoring systems originally developed and modified by Knaus and colleagues [8, 9]. To improve the accuracy of the APACHE method for predicting hospital mortality in critically ill adults and to evaluate increases in accuracy compared with earlier APACHE models, Zimmerman and associates [10, 11] developed the APACHE IV model for predicting hospital mortality.
The Sequential Organ Failure Assessment (SOFA) [12] was designed to describe morbidity. Although originally designed for classifying various degrees of organ failure rather than for predicting outcomes, various investigations have identified a clear relationship between organ dysfunction and mortality [13, 14].
The RIFLE classification (acronym indicating risk of renal failure, injury to the kidney, failure of kidney function, loss of kidney function, and end-stage renal failure) was first proposed by the Acute Dialysis Quality Initiative group to standardize study of acute renal failure (ARF) [15]. We reported the good discriminative power of the RIFLE category for predicting hospital mortality of critically ill patients treated with ECMO previously [6].
Given the promising new treatment methods and limited medical resources, investigators and physicians require reliable risk stratification and monitoring tools for patients during practice and clinical trials. Thus, this investigation compares the predictive ability of APACHE IV, earlier APACHE models, SOFA, and the RIFLE category obtained on the first day of ECMO support for hospital mortality in critically ill patients.
| Material and Methods |
|---|
|
|
|---|
Retrospective data included the following: demographics: primary diagnosis for ECMO implementation; whether or not the patient was currently being withdrawn from ECMO support; APACHE IV, APACHE III, APACHE II, SOFA scores, and RIFLE category on the first day of ECMO support; length of hospitalization; and outcome. The primary study outcome was hospital mortality rate. Follow-up at 6 months after hospital discharge was performed by means of chart record or telephone interview when necessary.
Definitions
Illness severity was assessed using APACHE IV, APACHE III, and APACHE II, which were calculated as described elsewhere [8–10]. Physiologic calculations used the worst physiologic values on the first day of ECMO support. When the patient was paralyzed or sedated, neurologic scoring was not performed, and the patient was not considered in neurologic failure. When a patient was intubated but not sedated, clinical judgment was used to estimate the best verbal response [7, 16]. A spreadsheet available at www.criticaloutcomes.cerner.com was used to calculate APACHE IV hospital mortality predictions. Diagnosis at admission to ICU and length of stay before ICU admission were recorded as primary reasons for ECMO support and duration of hospitalization before ECMO support, respectively.
Organ function was assessed using SOFA, which was defined as in the original report [12]. The most anomalous value of each of the six organ systems on the first day of ECMO support was also recorded. Patients were grouped with RIFLE classification according to risk, injury, and failure categories [15]. No patients met criteria for loss or end-stage renal disease categories. A simple mortality model was devised: non-ARF (0 points), RIFLE-R (1 point), RIFLE-I (2 points), and RIFLE-F (3 points) for day 1 of ECMO support [6].
Statistical Analysis
Continuous variables were summarized using means ± standard deviation unless otherwise stated. The primary analysis compared hospital survivors with nonsurvivors. All variables were tested for normal distribution using the Kolmogorov-Smirnov test. The Students t test was used to compare the means of continuous variables and normally distributed data; otherwise, the Mann-Whitney U test was used. Categorical data were tested using the
2 test. Correlation of paired variables within groups was assessed by linear regression using Pearson analysis. Finally, risk factors were assessed with univariate analysis, and variables that were statistically significant (p < 0.05) in the univariate analysis were included in multivariate analysis by applying a multiple logistic regression based on forward elimination of data.
Two methods were adopted to test calibration (ie, the degree of correspondence between predicted and observed mortality across the entire range of risk). First, calibration was graphically displayed by plotting observed and predicted mortality for all patients across all risk ranges. Second, goodness-of-fit testing was applied to determine calibration using the Hosmer-Lemeshow test [17].
Discrimination (ie, ability of the model to differentiate between mortality and survival) was evaluated by using the area under the receiver operating characteristic curve [18]. Sensitivity and specificity for APACHE IV, III, and II models, SOFA score, and RIFLE category were determined. Finally, cutoff points were calculated by deriving the best Youden index (sensitivity + specificity – 1) [6, 16, 19].
Cumulative survival curves as a function of time were determined by the Kaplan–Meier approach and compared using the log-rank test. All statistical tests were two-tailed; a probability value less than 0.05 was considered statistically significant. Data were analyzed using SPSS 12.0 for Windows (SPSS Inc, Chicago, IL).
| Results |
|---|
|
|
|---|
|
|
2 = 6.972, 8 degrees of freedom; p = 0.540) was superior to APACHE III (Hosmer-Lemeshow
2 = 22.013, 8 degrees of freedom; p = 0.005), and the APACHE IV calibration curve was closer to the line of perfect predictive ability than the calibration curve of the APACHE III model. Table 3
shows the goodness-of-fit, as measured by the Hosmer-Lemeshow
2 for predicted mortality risk, and the predictive accuracy of the APACHE IV, APACHE III, APACHE II, SOFA, and RIFLE scores. Table 3 also lists the discrimination for the APACHE IV, APACHE III, APACHE II, SOFA, and RIFLE scores. The discriminatory power of the APACHE IV score was superior to the APACHE III, APACHE II, SOFA, and RIFLE scores.
|
|
Hospital Mortality and Short-Term Prognosis
To assess the predictive value of each measure for hospital mortality, the sensitivity, specificity, overall correctness of prediction, and positive and negative predictive values were determined. The APACHE IV was found to have the best Youden index and highest overall correctness of prediction (Table 4). Hospital mortality rates differed significantly (p < 0.001) below and above cutoff points of 49% APACHE IV mortality rate, 91 APACHE III points, 22 APACHE II points, 13 SOFA points, and the "Risk" category of the RIFLE classification. Figure 2
illustrates that cumulative survival rates differed significantly (p < 0.001) between APACHE IV mortality rate (
49%) and APACHE IV mortality rate (>49%) in the study population.
|
|
|
| Comment |
|---|
|
|
|---|
Zimmerman and colleagues [10, 11] developed the APACHE IV model to improve the accuracy of earlier APACHE models. Studies using APACHE III, Simplified Acute Physiology Score II, and Mortality Probability Models II within independent ICU databases have reported a significant difference between predicted and observed mortality. These differences between observed and expected mortality might have been caused by poor model design, variations in quality of care, or inadequate case mix–related adjustment. Proposed reasons for inadequate case mix adjustment have included inadequate diagnostic data, unreliable Glasgow Coma Scale score assessment, international and regional differences, variations in patient referral patterns, and differing selection for and timing of ICU admission. Additionally, predicted outcomes are likely to be influenced by changes in the effectiveness of therapy as a function of time, the frequency of decisions to forgo life-sustaining therapy, care before and after ICU admission, and the frequency of early discharge to skilled nursing facilities [10]. In this study, the calibration of APACHE IV was superior to that of APACHE III (Table 3), thus indicating that the prognostic accuracy of APACHE IV is superior to APACHE III. In patients supported by ECMO, hospital survival declined and mortality was progressively underestimated by APACHE III (Fig 1B) but not APACHE IV (Fig 1A). Furthermore, computation of the area under the receiver operating characteristic curve verified the excellent discriminatory power of the APACHE IV and APACHE III models for day 1 of ECMO (Table 3). Finally, linear regression was applied to analyze the relationship between APACHE IV and APACHE III. An extremely significant correlation (r 2 = 0.902; p < 0.001) indicated that the two systems present similar evaluations despite differences in individual definitions between the two descriptors.
Acute renal failure, a manifestation of multiple organ system failure, is associated with underlying decompensated heart failure and sepsis and is aggravated by complications such as surgical site bleeding during ECMO support [6, 21]. Patients who experience ARF have high rates of mortality and resource utilization. Emerging evidence suggests that even small increases in creatinine levels after cardiac surgery are associated with significantly increased mortality. Whether ARF directly produces adverse outcomes remains unclear; however, increased infection and new-onset sepsis, congestive heart failure, and fluid overload may contribute to ARF [22–27]. Past criteria used to define ARF were an absolute cutoff value for serum creatinine and oliguria. However, as some patients with chronic renal dysfunction received diuretics, or had systemic hemodynamic changes, such criteria must have several stepwise cutoff values for serum creatinine and urine output to accompany its increment. The RIFLE criteria provide diagnostic definitions for the stage at which kidney injury can be prevented (risk stratum), when the kidney has been damaged (injury), and when renal failure occurs (failure) [28]. The RIFLE criteria have also been tested in clinical practice and seem to be at least coherent with regard to outcome of patients with acute kidney injury [6, 29–33]. As demonstrated in our previous [6] and present studies, this at least partly explains why RIFLE criteria can precisely predict hospital mortality (Tables 3, 5) in this subset of critically ill patients on ECMO day 1. The influence of other factors (eg, advanced age, type of surgery, history of chronic disease, hemodynamics, neurologic factors, or respiratory factors) on the morbidity and mortality of critically ill patients on ECMO are not measured in the RIFLE score. The failure to measure such extrarenal variables in RIFLE classification may explain its inferiority to APACHE IV, APACHE III, APACHE II, and SOFA scores in discriminative capability (Table 3).
In our hands, although an APACHE IV score greater than 49% was invariably associated with an extremely high in-hospital mortality and poor short-term prognosis, not enough patients were studied in this group to conclude that ECMO is futile. Physicians should not use such an arbitrary criteria as prognostic scores or factors to exclude patients from life-sustaining treatment; doing so would lead to a poor survival rate and hinder progress when caring for severely ill ICU patients [34].
Despite the promising results of this study, several important limitations must be recognized. First, this is a retrospective study performed at a single tertiary care medical center, which limits generalization of its findings. Second, difficulties exist by performing a retrospective study of this kind because some laboratory data were not available for all times, such as serum albumin, total bilirubin, and lactate. Third, we examined scoring systems only during the first day of ECMO support, although these models were developed and calculated on the first day of ICU admission. Finally, sequential measurement of these scoring systems (for example, daily, weekly) may reflect the dynamic aspects of clinical diseases, thus providing superior information on mortality risk.
In conclusion, in critically ill patients receiving ECMO support, a hospital mortality rate of 60.3% was observed in this study. The prognosis for these patients is poor. The RIFLE criteria classified 78.2% of critically ill patients receiving ECMO as having some degree of ARF. Analytical data also demonstrated the excellent discriminative power of APACHE IV in predicting hospital mortality of critically ill patients on ECMO support. This study suggests that APACHE IV can increase the accuracy of short-term prognosis in this subset of patients.
| References |
|---|
|
|
|---|
Related Article
This article has been cited by other articles:
![]() |
H.-S. Tsai, F.-C. Tsai, Y.-C. Chen, L.-S. Wu, S.-W. Chen, J.-J. Chu, P.-J. Lin, and P.-H. Chu Impact of Acute Kidney Injury on One-Year Survival After Surgery for Aortic Dissection Ann. Thorac. Surg., November 1, 2012; 94(5): 1407 - 1412. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Heilmann, G. Trummer, M. Berchtold-Herz, C. Benk, M. Siepe, and F. Beyersdorf Established markers of renal and hepatic failure are not appropriate to predict mortality in the acute stage before extracorporeal life support implantation Eur J Cardiothorac Surg, July 1, 2012; 42(1): 135 - 141. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y.-C. Chen, F.-C. Tsai, C.-H. Chang, C.-Y. Lin, C.-C. Jenq, K.-C. Juan, H.-H. Hsu, M.-Y. Chang, Y.-C. Tian, C.-C. Hung, et al. Prognosis of Patients on Extracorporeal Membrane Oxygenation: The Impact of Acute Kidney Injury on Mortality Ann. Thorac. Surg., January 1, 2011; 91(1): 137 - 142. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. R. Davies Invited commentary Ann. Thorac. Surg., October 1, 2007; 84(4): 1262 - 1263. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |