| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
a Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, the Netherlands
b Department of Public Health, Erasmus University Medical Center, Rotterdam, the Netherlands
c Department of Health Policy, Management and Behavior, School of Public Health, State University of New York, Albany, New York
Accepted for publication November 26, 2007.
* Address correspondence to Dr van Gameren, Department of Cardiothoracic Surgery, Bd 571, Erasmus University Medical Center Rotterdam, PO Box 2040, Rotterdam ZH, 3000 CA, the Netherlands (Email: m.vangameren{at}erasmusmc.nl).
 |
Abstract |
|---|
 Top Abstract IntroductionMaterial and MethodsResultsCommentReferences |
|---|
Methods: Required model variables were collected prospectively for all patients, followed by calculation of predictive mortality rates using the logistic and additive EuroSCORE, the logistic and additive NY State models for valve surgery without concomitant coronary surgery (isolated valve surgery) and the logistic and additive NY State models for combined valve and coronary surgery.
Results: Observed mortality was 2.8% (25 of 904) for isolated valve surgery and 6.8% (27 of 395) for valve plus coronary surgery. Logistic NY State and EuroSCORE expected mortality for isolated valve surgery was respectively 3.0% and 6.1%, and for valve plus coronary surgery 5.9% and 7.8%. The logistic NY State model for isolated valve surgery showed better discrimination (c-index 0.86 versus 0.76) and calibration than the logistic EuroSCORE. Discriminatory power for the logistic NY State model for valve plus coronary surgery was comparable to the logistic EuroSCORE (c-index 0.74 versus 0.72), as was calibration.
Conclusions: Our results suggest that dedicated risk models for valve surgery may be useful to provide more valid estimates of hospital mortality after heart valve surgery. Further exploration is needed to demonstrate general applicability of our results and assess the possible additional value of separate models for isolated valve surgery and valve plus coronary artery surgery, or aortic and mitral valve surgery, or both.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
The EuroSCORE model serves as a general cardiac surgery risk stratification model, while within the cardiac surgery population, there is a wide variety of procedures with different determinants of early mortality. Based on a dataset with mainly coronary surgery patients and only 30% valve surgery patients [3], EuroSCORE therefore may not be an optimal predictive model for valve surgery patients. Recently, several new models to predict early mortality after valve surgery have been developed that show an adequate to good performance [4–9]. However, while EuroSCORE may not be ideal to model early mortality after valve operations, the use of multiple models for different types of adult cardiac surgery may be more time and resource consuming, and may perhaps not have sufficient added value compared with the use of one common risk model.
Therefore we sought to answer the question whether in our institution a valve-dedicated risk stratification model would provide improved hospital mortality risk prediction for patients undergoing valve surgery. This was done by validating two recently developed valve-dedicated risk stratification models—a model for valve surgery without concomitant coronary surgery (isolated valve surgery) and a model for valve surgery with concomitant coronary surgery (valve plus coronary surgery) from New York (NY) State [4]—and comparing their performance with the EuroSCORE model.
|
Material and Methods |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
All collected variables were compliant with published EuroSCORE and NY State model variable definitions [2, 4]. Variables that initially did not meet all model definitions were all converted. EuroSCORE and NY State variables use different binary creatinine level and age cutoff points. Because we stored these as continuous variables, both definitions could be met. Myocardial infarction and surgery interval was also stored as a continuous variable, enabling a correct binary conversion for all models. Systolic left ventricular function (LVF) was measured qualitatively (good, impaired, moderate, and bad), thus complying with the EuroSCORE definition but deviating from the NY State model definitions (ejection fraction [EF] percentage). It was therefore assumed that good LVF equals EF of 40% or greater, impaired LVF equals EF 30% to 39%, moderate LVF equals EF 20% to 29%, and bad LVF equals EF less than 20%.
For this study, all patients who underwent surgery between January 2003 and January 2007 were selected. To meet all investigated model requirements, we excluded patients with concomitant ascending aortic surgery, aortic valve repair surgery, isolated tricuspid surgery, or isolated pulmonary valve surgery. Whenever patients underwent multiple operations within 1 month or during the same hospital admission, only the first (index) procedure was included. The data were stored in a local cardiac surgery database. Two patient groups were created from this database: patients who underwent isolated valve surgery were separated from patients who underwent valve surgery with concomitant coronary surgery. For all patients, predictive mortality rates were calculated using the published logistic and additive EuroSCORE coefficients [10]. Additional predictive mortality rates were added using the published NY State logistic and additive model coefficients [4] for their corresponding target group.
The dependent variable in this study was hospital mortality.
Statistical Analysis
Continuous data are presented as mean ± 1 SD, and median. Categorical data are presented as proportions. To evaluate the performance of risk models, we tested both discrimination (resolution) and calibration (reliability).
Discriminatory power was assessed using the c-index (area under the receiver operating characteristic [ROC] curve) with 95% confidence limits (CI). A c-index of 1.0 would indicate perfect discrimination, whereas a c-index of 0.50 indicates total absence of discrimination. A value between these extremes is the quantitative measure of a model’s ability to distinguish between survivors and nonsurvivors. To demonstrate significant differences between c-indices, a bootstrapping cycle of 2,000 runs was performed [11]. Tests between c-indices were two-sided with p less than 0.05 considered to be a significant difference.
Calibration was evaluated by the Hosmer-Lemeshow goodness-of-fit test and graphically by a calibration plot. The dashed smooth curve in a calibration plot reflects the nonparametric relation between observed mortality and predicted probability of mortality. Perfect calibration is represented by the straight dotted line through the origin. Triangles are based on quintiles of patients with similar predicted probabilities. Spikes at the bottom of a calibration plot represent the distribution of predicted probabilities. Models with Hosmer-Lemeshow p values above 0.05 were considered to be calibrated well for our population.
Descriptive statistical analyses were performed with SPSS version 13.0 (SPSS, Chicago, Illinois) and R version 2.5.1 (R Foundation for Statistical Computing, Vienna, Austria) was used for bootstrapping, calculating c-values with 95% CI, Hosmer-Lemeshow p values, and constructing ROC curves and calibration plots.
|
Results |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
|
Isolated Valve Surgery
Predicted mortality
The NY State logistic risk model for isolated valve surgery predicted a hospital mortality of 3.0% (95% CI: 2.7 to 3.3) versus a 2.8% observed rate for all isolated valve surgery patients. The NY State additive model predicted a mortality of 3.4% (95% CI: 3.1 to 3.7) for this group, the logistic EuroSCORE predicted a mortality of 6.1% (95% CI: 5.6 to 6.6), and the additive EuroSCORE predicted a mortality of 5.3% (95% CI: 5.1 to 5.5).
Discrimination
Based upon discriminatory performance in the isolated valve surgery group, ROC curves (Fig 1) were generated for the logistic and additive NY State and EuroSCORE models. Table 2
presents the corresponding c-indices. The c-indices of the NY State models are significantly higher than the EuroSCORE model c-indices (0.86 and 0.86 versus 0.74 and 0.76; all p < 0.05), demonstrating a better discriminatory power for the NY State models in our population.
|
|
|
Discrimination
When comparing the c-indices of the NY State models for valve plus coronary surgery to the EuroSCORE model c-indices (see Table 3), no significant differences were found in the valve plus coronary surgery group; none of the models outperformed the others regarding discrimination. A visual representation using ROC curves is given in Figure 3.
|
|
|
|
Comment |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
Rotterdam and New York State Population
When comparing the Rotterdam population with the one that the NY State models were based upon, only a few differences were found. Except for a higher prevalence of diabetes mellitus (valve plus coronary surgery group: 13.7% versus 29.0%), cerebrovascular disease (valve plus coronary surgery group: 6.1% versus 22.6%), and chronic obstructive pulmonary disease (isolated valve surgery group: 10.1% versus 18.5%) in the NY State population, both populations appear very similar.
Observed hospital mortality rates were slightly higher for the NY State population compared with the Rotterdam population for both the isolated valve surgery patient groups (4.4% versus 2.8%) and the valve with concomitant coronary surgery groups (8.9% versus 6.8%). Nevertheless, the NY State isolated valve surgery models only provide a slight overestimate of observed mortality in the Rotterdam population (3.0% and 3.4% versus 2.8% observed), whereas the NY State valve plus coronary surgery model slightly underestimated observed mortality in the Rotterdam dataset (5.9% and 6.2% versus 6.8%).
Apart from some small differences in patient characteristics between NY State and Rotterdam and the observed mortality differences, no evident explanation from our data can be found for the fact that the NY State valve with concomitant coronary surgery model does not outperform the EuroSCORE like the NY State isolated valve model does.
Risk Factors
In the past 5 years, several new models to predict early mortality after valve surgery have been developed that show an adequate to good performance [4–9]. Table 4
gives an overview of these models and their included risk factors, along with the EuroSCORE model. Many risk factors known to be relevant to predict mortality after valve surgery are included in the EuroSCORE [2, 12]. Several other important factors like diabetes mellitus and valve position, however, are missing in this model [4–9]. We assume a correlation between diabetes mellitus and factors that are included in the EuroSCORE model, such as extracardiac arteriopathy, is responsible for the absence of diabetes as a risk factor in this model.
|
Emergency surgery is a well-known predictor of higher mortality rates, and the EuroSCORE model does include emergency procedures as a risk factor, as do all other recently published valve-dedicated models. However, in both NY State models, no direct risk factor for emergency procedures was included. This was explained by a correlation between emergency status and shock or hemodynamic instability, factors that are included in the NY State models [13].
Furthermore, all recently developed models mainly use comorbidities as risk factors to predict procedure mortality. Cardiac and valve anatomy are however considered to be of even more importance in predicting procedure success, but are absent from almost all current valve risk models, except the NY State model for valve plus coronary surgery [14], which includes "severely calcified ascending aorta" as a risk factor.
Early Mortality
The dependent variable in both the Euroscore and NY State model is hospital mortality. One can debate whether this is an appropriate endpoint. It is much easier to collect patient information only during hospital stay and most other models also use hospital mortality; but on the other hand, postoperative hospital stay varies widely between centers and, in addition, it is well known that an increased post–cardiac surgery mortality risk does not disappear at patient discharge but continues for months after the procedure [15]. The new guidelines for reporting morbidity and mortality after cardiac valve interventions recommend that early mortality should be reported at 30, 60, and 90 days (submitted to the American Association for Thoracic Surgery [AATS], the European Association for Cardiothoracic Surgeons [EACTS], and The Society of Thoracic Surgeons [STS] for final review). Therefore, it is advisable that future valve models focus on a time-related endpoint independent of the location of the patient.
Even though all recently developed valve-dedicated risk stratification models are designed to predict short-term mortality only, they are probably able to predict 1 year (and possibly even longer) mortality as well. General cardiac surgery risk stratification models that were designed to predict short-term mortality have been demonstrated to possess this quality as well [16]. Future research is necessary to confirm this hypothesis. It would also be interesting to assess valve model performance in predicting other quality related outcomes. Again, risk stratification models that were built to predict short-term mortality after general cardiac surgery have been successfully used to predict morbidity and intensive care unit stay [17] and total length of stay [4]. Xu and coworkers [18] already published a model that successfully predicts intensive care unit stay after valve surgery (c-index 0.76; Hosmer-Lemeshow p = 0.25). Taking into account these facts, models included in Table 4 are likely to possess this quality as well.
Concomitant Coronary Surgery
Of all recently published valve-dedicated models (see Table 4), only the NY State model developers offer a separate model for isolated valve surgery and valve surgery with concomitant coronary surgery. In 2001, Edwards and associates [19] also developed separate models for isolated valve surgery and valve plus coronary surgery using the STS database.
The developers of all other recently published models have not employed this approach and simply included concomitant coronary surgery (or other type of surgery) as a risk factor in their valve models.
The reason for the development of a separate model for isolated valve surgery and a model for valve plus coronary surgery, according to Hannan and colleagues [4], lies in the substantially different patient characteristics between both groups, an observation that is confirmed in the Rotterdam population.
For example, mitral valve repair is a lower risk procedure than aortic valve replacement in the isolated valve model while mitral valve repair combined with coronary surgery is a higher risk procedure than aortic valve replacement with concomitant coronary surgery using the valve plus coronary surgery model, illustrating the difference between degenerated or myxomatous mitral valves in patients who need valve surgery only and the ischaemic nature of mitral valves in patients requiring concomitant coronary surgery.
Aortic and Mitral Surgery
Some model developers offer a separate model for aortic valve surgery and mitral surgery [8, 9]. This way, more significant risk factors can be distilled from these separate, more homogeneous groups. Related to the pathophysiologic differences between these groups, are differences between early mortality rates. As supported by the STS cohort, the models for mitral valve surgery produce higher predictive mortality rates [14]. However, models for aortic and mitral valve surgery have many similar risk factors [6].
Furthermore, in case of surgery on multiple valves, these models do not apply. A model that includes valve position(s!) as a model variable might therefore be preferred.
EuroSCORE, a Vintage Model?
Apart from both NY State models being valve-dedicated models, their development was undertaken on a 2001 to 2003 dataset, while EuroSCORE was based on data collected in 1995. Since the introduction of the EuroSCORE model, increasing procedure safety and decreasing mortality rates probably contribute to the better performance of the NY State isolated valve model in our population. For this reason, an improvement could also have been expected for the NY State valve plus coronary surgery model.
In this age of ubiquitous computing power, the simplicity of additive models might not justify their existence. And even though the EuroSCORE developers nowadays consider their additive EuroSCORE model to be inferior to the logistic model [10], we were not able to demonstrate a different performance between the logistic and additive EuroSCORE.
At the time of writing, the developers of the EuroSCORE just launched their plan to update the EuroSCORE in 2008 to reflect contemporary practice [20]. This new model will probably predict mortality after valve surgery better than the current EuroSCORE model owing to improvement of the calibration of a new EuroSCORE model. However, it may only outperform the NY State model for isolated valve surgery when specific factors that are more predictive of hospital mortality after valve operations, are included.
Study Limitations
This study is limited by the small sample size of the Rotterdam population, especially the valve plus coronary surgery group. Even though the patient group sizes were sufficient to perform decent statistical analyses, a larger population will result in more reliable model performance results. Another study limitation is its single center approach. Hospital and surgeon related factors might have resulted in study results that are not applicable to other centers. However, the purpose of this study was to evaluate model performance and usability in our center.
Other models from Table 4 were not included in our comparison because many risk factors that are required for these models were not available. Retrospective collection of these extra model variables would contaminate the prospectively collected data we used for the current comparison.
Valve Model Implementation
Firstly, the importance of clinical judgement should never be underestimated and regarding individual preoperative mortality prediction, our search has provided us merely with a new tool to support clinical decisions regarding valve surgery. A very useful tool though, with a use not limited to preoperative risk prediction only.
The question remains whether a center needs a general model that has been based upon an enormous population and was validated throughout a particular region or a smaller model constructed by only using its own patients. Both types have advantages that cannot be combined into one model to fit all needs.
Perhaps centers should use two separate valve-dedicated models; one model tailored to the center to predict actual and individual patient mortality and study the impact of different risk factors on outcome, and one model based upon a national or continental population, like the STS-derived models, to provide a benchmark for hospital performance. This way, all possible purposes for valve-dedicated risk stratification models are covered.
Noteworthy is that not only population selection is an important consideration when developing models for a specific purpose. For instance, the NY State risk stratification models were developed to serve as a tool to evaluate surgeon performance. By including the risk factor "extensively calcified ascending aorta," best measured by palpation during surgery, in the valve plus coronary surgery model, this model is less suitable for preoperative risk prediction purposes.
It is important to keep in mind that treatment for cardiac valve disease is still continuously improving and the danger of outdated models is considerable. In addition to periodic recalibration, future models will therefore need to be subject to improvements made by adjusting population sample size and reevaluation of risk factor inclusion and weights. The biggest step forward in model use should consist of implementing risk stratification models into the clinical process, whereby we use an optimal model for a prespecified patient population not only to monitor but also to improve outcome of our patients.
In conclusion, we demonstrated that in our clinical practice (even in a relatively small population) there is an added value using a valve-dedicated model like the NY State model in the setting of preoperative risk prediction and assessment of the impact of individual risk factors. We encourage other centers to conduct comparable studies to verify our results and increase general applicability. Several valve-dedicated models actually consist of two submodels [4, 8, 9]. Further exploration is needed to assess whether these separate models for isolated valve surgery and valve plus coronary surgery, or aortic and mitral valve surgery are really needed or whether one common valve model for each clinical purpose will suffice. With the emergence of less invasive valve repair and replacement techniques, the implementation and further development of risk stratification models in valve surgery will be of increasing importance in optimizing patient treatment.
|
References |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
Related Article
This article has been cited by other articles:
|
|
 |
|
  M. van Gameren, N. Piazza, A. J J C Bogers, J. J M Takkenberg, and A P. Kappetein How to assess risks of valve surgery: quality, implementation and future of risk models Heart, December 1, 2009; 95(23): 1958 - 1963. [Full Text] [PDF]
|
|
|
|
 |
|
 E. Manasse Cooks and recipes Eur. J. Cardiothorac. Surg., November 1, 2009; 36(5): 787 - 790. [Full Text] [PDF]
|
|
|
|
 |
|
 D. M. Shahian, S. M. O'Brien, G. Filardo, V. A. Ferraris, C. K. Haan, J. B. Rich, S.-L. T. Normand, E. R. DeLong, C. M. Shewan, R. S. Dokholyan, et al. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 3--valve plus coronary artery bypass grafting surgery. Ann. Thorac. Surg., July 1, 2009; 88(1 Suppl): S43 - S62. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. H. Rahimtoola The Year in Valvular Heart Disease J. Am. Coll. Cardiol., May 19, 2009; 53(20): 1894 - 1908. [Full Text] [PDF]
|
|
|
|
 |
|
 A. Vahanian, B. Iung, L. Piérard, R. Dion, and J. Pepper CHAPTER 21 Valvular Heart Disease ESC Textbook of Cardiovascular Medicine, January 1, 2009; 2(1): med-9780199566990-chapter - med-9780199566990-chapter. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. G. Byrne, M. Leacche, D. E. Vaughan, and D. X. Zhao Hybrid Cardiovascular Procedures J. Am. Coll. Cardiol. Intv., October 1, 2008; 1(5): 459 - 468. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
F. Edwards Invited Commentary Ann. Thorac. Surg., March 1, 2008; 85(3): 930 - 931. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
