|
|
||||||||
a Department of Surgery, Erasmus Medical Center, Rotterdam, The Netherlands
b Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, Amsterdam, The Netherlands
c Department of Surgery, Academic Medical Center, Amsterdam, The Netherlands
Accepted for publication June 7, 2010.
* Address correspondence to Dr Grotenhuis, Erasmus Medical Center, Department of Surgery, PO Box 2040, 3000 CA Rotterdam, The Netherlands (Email: b.grotenhuis{at}erasmusmc.nl).
| Abstract |
|---|
|
|
|---|
Methods: An independent dataset of 777 patients who underwent esophagectomy for cancer was used for validation. The discriminatory capability of the nomogram was determined by using the concordance index (C statistic). Calibration was evaluated by comparing the observed with the expected number of patients with complications, as predicted by the original nomogram across patients with different risk profiles. We also examined whether adjusting the value of the original coefficients of the predictors or adding new predictors would improve the fit of the nomogram.
Results: Discrimination of the original nomogram was similar in the validation cohort: the C statistic hardly decreased from 0.65 in the original cohort to 0.64 in the validation cohort. Observed and expected number of patients with complications were in close agreement, reflecting a good calibration (p = 0.84). Reestimation of the coefficients in the validation cohort did not lead to any significant changes of the original nomogram values.
Conclusions: External validation of a nomogram predicting the occurrence and severity of complications after esophagectomy showed that the model is applicable in other high-volume hospitals. Nevertheless, preoperative prediction of complications in individual patients remains difficult, most likely due to the complexity of mechanisms causing these complications.
| Introduction |
|---|
|
|
|---|
Predicting the severity of complications after esophagectomy may supply important information for both patient and surgeon. The impact of postoperative complications on quality of life plays an important role in decision making whether to proceed with an operation in an individual patient with esophageal cancer. Furthermore, the prediction of severity of complications in the preoperative phase may help in choosing the extent of the operation and in informing patients. Recently, a nomogram has been developed to predict the occurrence and severity of complications in esophagectomy patients [12]. A nomogram gives a graphic representation of the predictive strength of specific predictors and enables clinicians to calculate an overall risk score for individual patients reflecting their personal risk. In this nomogram the severity of complications was predicted at three levels: no complications, minor complications, or severe complications. Specific patients' characteristics (ie, more advanced age, myocardial infarction, stroke or transient ischemic attack in the medical history, lower forced expiratory volume in one second, and presence of electrocardiographic changes) and the application of more extensive surgery (ie, transthoracic esophagectomy [TTE]) were associated with a higher risk of (more severe) postoperative complications.
This new nomogram has been validated in a second group of patients within the same institution (temporal validation), which did not reveal any statistically significant changes in the predictive strength of the included prognostic factors. However, the power of that validation study was only moderate given the limited sample size of 100 patients [13], and validation within the same institute may not reveal inherent problems of a prognostic model that can become apparent when differences in patient populations, surgeons, applied surgical techniques, and postoperative care between institutions are taken into account [14]. Therefore, the aim of the present study was to externally validate this nomogram in a new and large cohort of patients who underwent esophagectomy for cancer in a different high-volume center.
| Patients and Methods |
|---|
|
|
|---|
The Medical Ethics Committees of the participating hospitals that provided the derivation cohort (Academic Medical Center, Amsterdam) and the validation cohort (Erasmus Medical Center, Rotterdam) have approved the ongoing registry of data of esophageal cancer patients in prospective databases. They also approved the publication of the data used for the current study.
Definitions of Complications
The severity of postoperative complications was graded according to the morbidity scale proposed by Dindo and colleagues [18]. This classification system is based on the therapeutic consequences of complications and consists of five grades and two subgrades. Grade I complications do not need any medical or surgical intervention (eg, atelectasis, vocal cord paralysis, radiologic anastomotic leakage). Grade II complications need pharmacologic treatment (eg, pneumonia, chyle leakage, pulmonary embolus). Grade III complications need an intervention (grade IIIa: eg, anastomotic leakage requiring drainage of the neck, pneumothorax, or pleural empyema requiring drainage of the chest; grade IIIb: any reoperation). Grade IV complications are life threatening and represent single organ failure (grade IVa: eg, pulmonary dysfunction requiring artificial ventilation, heart failure requiring intravenous inotropic agents, renal insufficiency requiring dialysis) or multiorgan failure (grade IVb). Finally, grade V complications are complications leading to death. Grading of complications was performed according to the most severe complication in each patient. Similar to the original study, complications were categorized into three groups: no complications, minor to moderate complications (grades I to IIIb), and severe complications (grades IVa, IVb, and V).
Statistical Analysis
All analyses were performed using SAS software version 9.1 (SAS Institute Inc, Cary, NC). Clinicopathologic characteristics of the derivation cohort (Academic Medical Center, Amsterdam) and the validation cohort (Erasmus Medical Center, Rotterdam) were analyzed in a descriptive way. Data on daily alcohol intake and presence of preoperative dyspnea were not available for the Rotterdam patients. In Rotterdam, weight loss was classified as a categoric value rather than a continuous variable (Amsterdam).
Ordinal logistic regression was used to examine the association between predictors and the occurrence of complications classified in three categories of severity (no complications, minor complications, severe complications). The ordinal logistic regression (proportional odds model) is an extension of the binary logistic regression model that is used in case of three or more outcome states that are naturally ordered [19]. Univariate analyses of potential predictors for the severity of complications were performed with the data set of the Rotterdam cohort. Because missing data result in loss of statistical power and can lead to possible bias, multiple imputation techniques were applied [20]. All predictors, as well as the observed outcome, were used to impute missing values based on multivariate normal distributions using the Markov chain Monte Carlo method. The coefficients of ten rounds of imputations were combined to obtain the final estimates of odds ratios and their 95% confidence intervals.
The nomogram predicting the occurrence and severity of complications previously developed in Amsterdam was validated in several ways on the Rotterdam patient cohort. First, discrimination was evaluated by calculating a risk score for each patient in the validation cohort based on the original coefficients from the derivation cohort. The discriminatory properties of the model were examined by visualizing the distribution and overlap in risk scores of individual patients within and between the three outcome categories. Furthermore, the discriminative capability was quantified by using the concordance (C) statistic. The C statistic is a measure that can be interpreted as the probability among all possible pairs between patients from different outcome categories that the patient with the more severe complication also has the higher risk score. Values can range from 0.5 (due to chance, no discrimination) to 1.0 (perfect discrimination). We calculated C statistics for the three categories (no complications versus minor complications, minor complications versus severe complications, no complications versus severe complications) as well as the overall C statistic in the validation cohort. These data were compared with the original C values in the derivation cohort.
Second, calibration was evaluated by comparing the expected and observed number of patients in each of the three outcome categories across seven quantiles of expected risk (ie, seven groups of each 111 patients with an increasing mean risk score). Calibration was tested for significance by using an extension of the Hosmer-Lemeshow goodness-of-fit statistic [21].
The third analysis examined whether the importance of the individual predictors or intercepts within the original nomogram was different in the validation cohort. For each predictor a proportional odds model was fitted with only that predictor using the risk score based on the original coefficients as an offset variable. Ideally, if the weight (importance) of the predictor is comparable between the validation cohort and the derivation cohort, the coefficient of each predictor would be zero as its weight is already incorporated in the risk score. The likelihood ratio test was used to indicate whether reweighing of the predictor will significantly improve the model. Similarly, potential predictors that did not significantly improve the prognostic performance of the original nomogram were reevaluated in the derivation cohort whether they improved the prognostic performance of the model in the new cohort.
| Results |
|---|
|
|
|---|
|
|
|
|
|
| Comment |
|---|
|
|
|---|
The clinicopathologic characteristics of patients undergoing esophagectomy for cancer in the validation cohort in Rotterdam and the derivation cohort in Amsterdam were comparable with regard to potential prognostic factors such as age, American Society of Anesthesiologists classification, body mass index, and comorbidities (Table 1). However, the median preoperative risk score as calculated by the original nomogram was significantly lower in Rotterdam than in Amsterdam. Apparently, the preoperatively determined risk factors were more prevalent in Amsterdam. Another explanation could be the fact that a TTE was performed more often in the original cohort (36% Amsterdam versus 4% Rotterdam). The weight or importance of the predictor operation technique in the validation cohort did not significantly differ from that in the original nomogram, but it can be questioned whether the power of this parameter in the validation cohort was sufficient to evaluate this issue adequately. Overall, no large differences in grades of complications impeded an adequate evaluation of the two patient cohorts in this study (Table 2), although the comparison of occurrence and severity of complications between hospitals is subject to bias because of variable definitions of complications and different scoring systems.
When analyzing the discriminatory capability of the original risk score in the validation cohort, it appeared that the mean risk scores in the three outcome categories were significantly different, but a substantial overlap in scores among the three categories was noted (Fig 1). This was also reflected in the pairwise C statistics among the three complication categories, and the overall moderate discriminative power of the model (0.64 in the validation cohort, 0.65 in the derivation cohort). Apparently, patient-related factors are not the only determinators responsible for developing complications after esophageal cancer surgery. The intraoperative course (eg, fluid management by the anesthetist [22, 23] and surgical complications resulting in a longer operative time or more blood loss [2]), as well as the early postoperative phase (eg, application and effectiveness of epidural analgesia [24–26]), will also influence the final outcome of patients who undergo esophagectomy for cancer. However, these factors cannot be predicted in a preoperative risk assessment.
Nevertheless, the overall fit of the model and the calibration of the nomogram score in the validation cohort (ie, the closeness of predicted and observed frequency of outcomes) were good; the predictions for groups of patients with a similar risk profiles matched the observed probabilities (Fig 2). The expected frequency of patients with no or minor complications was higher than expected on the basis of the nomogram, while these differences improved after adjusting the intercept (ie, background risk for the new cohort) suggesting that the general level of complications was lower in the validation cohort. One could hypothesize that this may reflect differences in postoperative care between the two hospitals (eg, more specialized intensive care unit-care and faster detection of complications, preventing an increase in complication grade). On the other hand it can be questioned whether the complication registration in the validation cohort was as efficient and complete as in the derivation cohort.
Despite the complexity of mechanisms that can lead to the development of complications (preoperative, intraoperative, and early postoperative factors, as well as intrinsic [patient-related] and extrinsic [hospital-related] elements) and the subjectivity in the registration of complications, the optimal coefficients of the prognostic factors in the validation cohort did not significantly differ from the coefficients of the original nomogram model. This indicates that the current nomogram is applicable in other high-volume hospitals performing esophagectomies for cancer. The model can give a preoperatively estimated risk of the occurrence and severity of postoperative complications. Furthermore, as we have shown that adequate model performance can be achieved, this model can be used to adjust for case-mix when comparing hospital performances; the nomogram can play a role in the risk-adjusted audit of morbidity after esophagectomy for cancer.
In conclusion, a recently developed nomogram predicting the occurrence and severity of complications was externally validated in a new cohort of patients who underwent esophagectomy for cancer in a different high-volume center. The model showed good overall calibration when applied in the validation cohort. Reestimating the coefficients of the prognostic factors within the nomogram in the validation cohort did not reveal significant improvement compared with the original values. Nevertheless, preoperative prediction of complications in individual patients remains difficult, most likely due to the complexity of mechanisms causing these complications.
| Acknowledgments |
|---|
|
|
|---|
| References |
|---|
|
|
|---|
Related Article
This article has been cited by other articles:
![]() |
M. E. van Genderen, A. Lima, H. de Geus, E. Klijn, B. Wijnhoven, D. Gommers, and J. van Bommel Serum C-Reactive Protein as a Predictor of Morbidity and Mortality in Intensive Care Unit Patients After Esophagectomy Ann. Thorac. Surg., June 1, 2011; 91(6): 1775 - 1779. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. K. Ferguson Invited Commentary Ann. Thorac. Surg., September 1, 2010; 90(3): 926 - 926. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |