|
|
||||||||
a Division of Cardiology, Veterans Administration Medical Center and University of Minnesota, Minneapolis, Minnesota
b Division of Cardiovascular Surgery, Veterans Administration Medical Center and University of Minnesota, Minneapolis, Minnesota
Accepted for publication April 1, 2011.
* Address correspondence to Dr Adabag, Veterans Administration Medical Center, Cardiology 111C, One Veterans Dr, Minneapolis, MN 55417 (Email: adaba001{at}umn.edu).
| Abstract |
|---|
|
|
|---|
Methods: We included 537 consecutive patients who underwent AVR for severe aortic stenosis at the Minneapolis VA Medical Center between 1997 and 2008. Observed and predicted perioperative (30-day) mortality rates were compared. Hosmer-Lemeshow goodness-of-fit test and receiver operating characteristic curves were performed to assess the performance of the scores.
Results: Perioperative mortality rate was 5.9% (n = 32). Predicted mortality rates for the EuroSCORE, STS score, and VA score were 15.6%, 3.6%, and 6.7%, respectively (p = 0.001). The EuroSCORE overestimated mortality in all patients, most notably among those with ejection fraction less than 35% (49% predicted versus 9% observed). The EuroSCORE had poor calibration (goodness-of-fit test p < 0.008), whereas the STS and the VA scores were well calibrated. However, all three scores displayed good discrimination characteristics per the areas under the receiver operating characteristic curves: STS score 0.73 (95% confidence interval: 0.69 to 0.77); VA score 0.66 (95% confidence interval: 0.62 to 0.70); and EuroSCORE 0.68 (95% confidence interval: 0.64 to 0.72; p > 0.05).
Conclusions: The EuroSCORE substantially overestimates perioperative mortality risk in AVR, particularly in patients with low ejection fraction. These data have implications when deciding the appropriate intervention (transaortic valve implantation versus AVR) for high-risk aortic stenosis patients.
| Introduction |
|---|
|
|
|---|
Perioperative mortality risk can be estimated by several clinically useful risk scores such as The Society of Thoracic Surgeons (STS) predicted risk of mortality score, the European System for Cardiac Operative Risk Evaluation (EuroSCORE), and the Veterans Administration (VA) risk score. These scores have been derived from analysis of clinical variables and outcomes in large cardiovascular surgery databases and have subsequently been validated in prospective studies [4-6]. Often, a single risk score is preferred over others based on personal, institutional, or geographical biases. However, an objective evaluation of the performance of these scores in predicting perioperative mortality after AVR has not been performed.
The objective of the present investigation was to compare the accuracy of the STS score, the EuroSCORE, and the VA score in predicting perioperative mortality in a large cohort of patients with severe AS who underwent AVR with or without concomitant coronary artery bypass graft surgery.
| Material and Methods |
|---|
|
|
|---|
Patient Population and Data Acquisition
We included 537 consecutive patients with severe AS who underwent AVR with or without coronary artery bypass graft at the Minneapolis VA medical center between 1997 and 2008. Patients who underwent AVR for aortic regurgitation, endocarditis, and aortic root disease, and those who had double valve surgery were excluded. Preoperative clinical variables were obtained from the cardiac surgery database at our medical center. This is part of a national database of prospectively collected data on all patients undergoing cardiac surgery within the VA medical centers [5, 7-9]. Echocardiographic data were obtained from electronic medical records.
Veterans administration risk score
Since 1987, data from all patients undergoing cardiac surgery at VA medical centers have been collected as part of a quality improvement program [8]. This ongoing, prospective database includes demographic information, preoperative clinical variables, surgical details, and postoperative outcomes including 30-day mortality and major complications. The database also includes a validated risk score, which measures patient risk at the time of cardiac surgery on a scale that ranges from 0% to 100%, with the higher numbers indicating greater risk [7, 8, 10, 11]. This score is calculated by multivariable logistic regression analysis where the variables with p less than 0.2 in univariable analysis are included in the multivariable logistic regression model. Twice each year, statistical analyses are done to assess the performance of each cardiac surgical center and to recalibrate the regression models that predict operative mortality.
Euroscore
The EuroSCORE was developed to assess the risk of mortality in adult patients undergoing cardiac surgery in Europe (12, 13) and measures patient risk at the time of cardiac surgery on a scale that ranges from 0% to 100%, with the higher numbers indicating greater risk. The dataset consisted of 68 preoperative and 29 operative risk factors with a potential impact on mortality. The EuroSCORE has subsequently been validated in larger cardiovascular surgery trials (4, 14). In the present study, additive and logistic EuroSCORE was calculated using the online calculator (http://www.euroscore.org/calc.html).
STS predicted risk of mortality score
The STS score consists of risk calculation for seven specific surgical procedures and measures patient risk at the time of cardiac surgery on a scale that ranges from 0% to 100%, with the higher numbers indicating greater risk. It is composed of 41 variables that are used to derive nine clinical endpoints. In the present study, the online calculator was used to calculate operative mortality for each patient based on his preoperative variables (available at: http://209.220.160.181/sts scorewebriskcalc261/de.aspx). The STS score is recalibrated every quarter.
Outcome
The primary outcome was operative mortality defined as death within 30 days of heart surgery due to any cause, or death at a later time occurring as a direct consequence of a perioperative complication. Follow-up was complete in all study patients.
Statistical Analysis
Continuous variables were described as mean ± SD and categorical variables as percentages. Predicted mortality rates by the three risk scores were compared using analysis of variance. Excess mortality was calculated by subtracting the observed from the expected mortality rate. The Hosmer-Lemeshow goodness-of-fit test was performed to assess the calibration of each risk score with the observed mortality [15]. Receiver operating characteristic (ROC) curves were created for each risk score to determine its ability to discriminate the patients who died from those who survived. The area under the ROC curve was presented as c-indexes and were compared by the method of DeLong [16]. A p value less than 0.05 was taken as statistically significant.
| Results |
|---|
|
|
|---|
|
|
|
Distribution of Risk Factors
The distribution of high-impact risk factors in each risk score is displayed in Table 3. The variables extracardiac arteriopathy and critical preoperative state, which were only present in the final EuroSCORE risk model, constituted 37% and 14%, respectively, of the overall study group, and 43% and 33%, respectively, of the patients with EF less than 35% (Table 3).
|
| Comment |
|---|
|
|
|---|
The predictive performance of risk scores is defined by two fundamental characteristics, calibration and discrimination [17]. Calibration assesses the closeness of the observed and predicted mortality rates. This is statistically evaluated by goodness-of-fit test. If the estimated and observed mortality are close, the score is considered well calibrated. Discrimination represents the ability of the score to distinguish patients who died from those who survived (or high risk versus low risk) without being encumbered by how close the predicted and actual risks are. Statistically, discrimination is assessed by the area under the ROC curve and ranges between 0 and 1.0. A test approaches excellent discrimination as the area under the ROC curve approaches 1.0. An ideal risk score should have both excellent calibration and discrimination.
The EuroSCORE has previously been noted to overestimate mortality rate in AVR. In prior reports, predicted mortality rate by EuroSCORE was almost 10% higher than the observed mortality among high-risk AVR patients [18] and 4% to 8% higher than the observed mortality among average-risk AVR patients [19-21]. The results were similar for additive versus logistic EuroSCORE [19, 20]. Further, in previous comparisons, the STS score has been noted to predict mortality more accurately than the EuroSCORE in AVR [6, 21]. Finally, in the recent Placement of Aortic Transcatheter Valves (PARTNER) trial [22], the mean EuroSCORE was substantially higher than the STS score (28 versus 12). The present investigation takes these data one step further by providing head-to-head comparison of the VA, STS, and EuroSCORE. It remains imperative that special attention is paid to the method for determining high-risk patients as the predicted mortality may be substantially overestimated.
The EuroSCORE was derived from operative data of nearly 15,000 patients undergoing cardiovascular surgery in 1995 [12, 13]. The additive model was developed in 1999, and a refined logistical model was presented in 2003 [23]. The EuroSCORE has not been recalibrated since then; however, a project to calculate a new EuroSCORE is on the way. The VA and the STS scores are recalibrated two and four times yearly. Indeed, every 6 months, new data from cardiac surgical procedures at VA medical centers are collected and incorporated into the national database. Then, statistical analyses are repeated using the most recent 3 years' data to refine regression models. This procedure recalibrates the VA score and allows new variables to be incorporated in the model in a relatively short time. We believe that the changing surgical practice dictates the risk scores to be recalibrated.
Another possible explanation for the differences in estimated risk between the EuroSCORE and the other two risk scores is the differences in the source populations from which the risk score has been derived. Indeed, the prevalence of the high-impact risk factors in each risk model might vary from one population to another (Table 3). It would be expected that the risk score performance is best in the population from which it was derived. Thus, the clinicians are advised to select a risk model derived from a population that most closely reflects their own patients.
Many preoperative variables are considered when calculating risk scores. However, factors such as nutritional status or predilection to infection cannot be measured or are not captured in databases. Thus, they are not included in risk score calculation. Some of the variables that are not included in statistical models might be accounted for in physicians' clinical assessment. Indeed, discriminative power increases when the statistically obtained risk score and the clinical assessment of the physician are considered together [24]. Further, intraoperative and postoperative factors such as hypotension, blood loss, infection, pressor use, and reaction to medications anesthesia cannot be taken into consideration in preoperative risk score calculation. However, these factors and other intraoperative and postoperative complications that increase the ischemic and cardiopulmonary bypass times increase perioperative morbidity and mortality [25, 26]. These unavoidable shortcomings are the likely cause of the limitation in the discrimination ability of the risk scores.
It is estimated that approximately 30% of patients with severe AS are not operated on because of high predicted mortality after AVR [27]. Recently, TAVI has been introduced as an alternative to AVR in these patients, and in some cases, an elevated EuroSCORE has been used as the benchmark criterion to document high-risk status [28]. However, cumulative data from the present and previous investigations show that the EuroSCORE overestimates mortality risk and should not be used in this context [18-21]. The STS score and the VA score are better suited for this purpose. Other risk scores to predict the outcome of valvular surgery are also present. Two such risk scores were recently developed and validated using the New York State cardiac surgery reporting system (29).
The strengths of this study include head-to-head comparison of three widely accepted surgical risk scores using a complete database with excellent follow-up and sound statistical methods. There are also notable limitations. First, the study was carried out at a single referral institution with a track record of favorable surgical results. Thus, the perioperative mortality rate was low, increasing the confidence intervals in the ROC analyses. Second, the patients with EF less than 35 represented 12% of the overall study cohort. A larger sample of this subgroup would have allowed more powerful subgroup analyses of this group of patients with highest mortality. Third, the participants were all men. Caution should be exercised in extrapolating these data to women.
In conclusion, the EuroSCORE substantially overestimates perioperative mortality risk after AVR and does not appear to be well suited for use when deciding between AVR versus TAVI for patients with AS. The STS score and the VA score, however, provide a more accurate prediction of mortality risk.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
M. Carnero-Alcazar, J. A. Silva Guisasola, F. J. Reguillo Lacruz, L. C. Maroto Castellanos, J. Cobiella Carnicer, E. Villagran Medinilla, T. Tejerina Sanchez, and J. E. Rodriguez Hernandez Validation of EuroSCORE II on a single-centre 3800 patient cohort Interact CardioVasc Thorac Surg, March 1, 2013; 16(3): 293 - 300. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Drews, M. Pasic, S. Buz, G. d'Ancona, S. Dreysse, M. Kukucka, A. Mladenow, R. Hetzer, and A. Unbehaun Transcatheter Aortic Valve Implantation in Very High-Risk Patients With EuroSCORE of More Than 40% Ann. Thorac. Surg., January 1, 2013; 95(1): 85 - 93. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. A. M. Nashef, F. Roques, L. D. Sharples, J. Nilsson, C. Smith, A. R. Goldstone, and U. Lockowandt EuroSCORE II Eur J Cardiothorac Surg, April 1, 2012; 41(4): 734 - 745. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |