|
|
||||||||
Ann Thorac Surg 1998;66:1254-1262
© 1998 The Society of Thoracic Surgeons
a Unit for Quality Assurance, The Gertner Institute for Epidemiology and Health Policy Research, Sheba Medical Center, Sackler School of Medicine, Tel Aviv University, Tel Hashomer, Israel
b Department of Social Medicine, School of Public Health, Jerusalem, Israel
Accepted for publication April 25, 1998.
Address reprint requests to Dr Mozes, The Gertner Institute for Epidemiology and Health Policy Research, Sheba Medical Center, Tel Hashomer 52621, Israel
e-mail: (benjamin{at}post.tau.ac.il)
| Abstract |
|---|
|
|
|---|
Methods. This study analyzed data from consecutive patients who underwent isolated coronary artery bypass grafting at 14 medical centers. Data included demographic and clinical information, comorbidity, cardiac catheterization results, and 30-day postoperative vitality status. Logistic regression analysis was used to identify variables associated with mortality. An outlier hospital was defined as one having an observed mortality outside the 95% confidence interval boundaries around the expected mortality rate calculated, given the patient risk factors.
Results. The overall crude 30-day mortality rate for isolated coronary artery bypass grafting among the 4,835 patients in this study was 3.1%. The rate varied among centers, ranging from 0.85% to 7.05%. Predictors of 30-day mortality included advanced age, female sex, diabetes mellitus, poor left ventricular function, high creatinine level, high priority of operation, and three-vessel disease (with or without left main coronary artery disease). After adjustment for risk factors, two hospitals were defined as outliers.
Conclusions. The observed disparity in early mortality among patients undergoing coronary artery bypass grafting is not due solely to differences in case mix.
| Introduction |
|---|
|
|
|---|
Health-care outcomes have been described as a function of patient attributes, effectiveness of care, and other factors, including random events [4]. Therefore, whenever the primary goal of analysis is to identify quality-of-care problems among centers, the main methodologic challenge is to eliminate as much of the variance stemming from patient factors as possible. The ability to do this is still a major concern for hospital management and for the medical community [5, 6].
Several studies have delineated the important case-mix variables that predict early mortality in CABG patients. However, few studies have used this model to compare risk adjustments among hospitals. Only two such initiatives included institutions of varying sizes and types (academic versus private hospitals) in their analysis [7, 8], and only The Society of Thoracic Surgeons [8] in the United States has attempted to produce a national database. Additional studies would be helpful by demonstrating the extent to which previously developed models can be generalized to other medical communities and by comparing the practice of CABG throughout the world: the overall mortality rate, the characteristics of patients undergoing CABG, and the variance among institutions.
The present study is a nationwide survey of all 14 centers in Israel performing CABG. The study was designed with the following two objectives: to create a risk model for early mortality in CABG operations suited to the Israeli surgical environment and to compare it with models developed in other settings; and to identify outlier centers among hospitals performing CABG in Israel.
| Material and methods |
|---|
|
|
|---|
Data gathering
In each center, a trained nurse collected the data using a structured questionnaire. To improve reliability of data collection, the nurses underwent intensive training for this task. After a short pilot study, ambiguous definitions were removed or changed on the basis of discussions with the nurses. To ensure that each institution entered every patient eligible, the study coordinators compared the operating book lists with the study participants in all hospitals. The operating books showed all patients who underwent operation with practically no missing data.
Data collection included preoperative direct patient interviews and a follow-up of vitality status using official national mortality records. The discharge sheets, operation reports, and catheterization reports were obtained for each patient and sent to the research center. The discharge abstract and the operation reports were coded by a trained general practitioner. Guided by a structured questionnaire, a cardiologist interpreted and coded the catheterization reports.
The data were initially entered using computer software that included internal range checks; unlikely values were thus rejected as the data were gathered. The quality of data was further maintained by random chart audits conducted by nurse coordinators at each site.
Data analysis
The preoperative risk factors used and the source of information for each are specified in Appendix 2. The variables included in the analysis were derived from clinical judgment and statistical considerations. Clinical judgment was used to determine the following: how to manage information emanating from two sources (patient interview versus chart extraction), either to ignore one of the sources or to define the existence of a risk factor as being present according to two sources, and how to combine information received in response to several questions to construct a clinically meaningful variable, eg, diuretics and shortness of breath to describe congestive heart failure.
Statistical means were used to perform: categorization of variables not associated with mortality in a linear or log-linear way (eg, creatinine level) and completion of missing data for ejection fraction (EF). The latter constituted a special problem, as EF is a crucial variable and the values were missing for nearly 20% of patients. We calculated the regression equation for EF on the basis of the EF data we had and used this equation to calculate a predicted EF for each patient without this item.
To determine the cut-off points of the categoric variables, we used clinical considerations (eg, EF < 0.40 as a marker of left ventricular function) and frequencies of patients in each category (eg, only a few patients in the study population had a creatinine level >2 mg/100 mL).
To determine which of the proposed risk factors were significantly associated with 30-day postoperative mortality, a pooled analysis (for the entire patient population) was performed: bivariate analysis of the association of each independent variable (logistic regression) with 30-day mortality, logistic regression analysis, and internal validation of the final logistic regression model that predicted the 30-day postoperative mortality rate. Variables were selected as candidates for the multivariate analysis on the basis of the level of significance of the bivariate association with 30-day mortality (p < 0.05); variables with more than 10% missing data were omitted.
The internal validation of the final logistic regression model was carried out in three steps. First, the area under the receiver operating characteristic (ROC) curve was examined to check the ability of the model to discriminate between living and dead patients. The ROC curve is a plot of the true-positive rate (on the vertical axis) and the false-positive rate (on the horizontal axis). The area under the ROC curve represents the probability that a random pair of living and dead persons will be correctly ranked as to their vitality status. Second, goodness-of-fit test of Hosmer and Lemeshow [9] was used to compare the expected and the actual number of deaths. The patients were divided into nine groups of roughly the same size on the basis of the percentiles of the estimated probabilities of death using the regression equation predicting mortality. The discrepancies between the observed and the expected number of deaths in these groups were summarized by the Pearson
2 statistic, which was then compared with a
2 distribution with t degrees of freedom, where t is the number of groups minus 2 [9]. Third, cross-validation was accomplished by randomly dividing the data into two sets, ie, a training set encompassing two thirds of the patients and a validation set with the remaining third. The model developed with the training set was tested on the validation set. The area under the ROC curve in the two sets was compared.
To compare the adjusted mortality rates among hospitals, we computed the expected mortality rate for each hospital by calculating the expected probability of a 30-day postoperative death for each patient in the study, given the patient risk factors, and then averaging the probabilities for all CABG patients in each medical center. A 95% confidence interval was calculated for the predicted mortality in each hospital. An outlier hospital was defined as one having an observed mortality outside the confidence interval boundaries.
| Results |
|---|
|
|
|---|
|
Crude mortality rates
The in-hospital mortality rate for the entire population was 2.35%. The 30-day postoperative mortality rate was 3.10%. The range in 30-day mortality rate among medical centers was 0.85% through 7.05%; 13.3% of the deaths occurred within 48 hours after operation.
Risk model
Bivariate analysis revealed that the preoperative patient characteristics associated with 30-day postoperative mortality were as follows: advanced age, female sex, diabetes mellitus, hypertension, peripheral vascular disease, past stroke, creatinine level higher than 1.4 mg/100 mL, high severity level of angina pectoris, clinical congestive heart failure, priority of operation (urgent and emergent), operation during acute myocardial infarction, extent of coronary artery disease (two or three obstructed vessels with involvement of left main artery), and low EF (see Table 1). Multivariate analysis of the demographic and clinical patient characteristics prior to operation showed the following variables to be associated with 30-day postoperative mortality: advanced age, female sex, diabetes mellitus, clinical congestive heart failure (defined by use of diuretics and shortness of breath), creatinine level higher than 1.4 mg/100 mL, EF lower than 0.40, priority of operation (emergent and urgent), and extent of coronary artery disease (three-vessel disease with or without left main disease) (Table 2).
|
|
Comparison among hospitals
Two hospitals were found to be negative outliers in our study, ie, their observed mortality rate was beyond the upper boundary of the confidence interval of their predicted mortality rates. In addition, one hospital was a borderline-positive outlier (Fig 1).
|
| Comment |
|---|
|
|
|---|
|
Despite major discrepancies among the various studies in the areas of inclusion criteria, surgical setting, and definitions of variables, the results are surprisingly similar. The following differences in research designs, and therefore results, are apparent:
In summary, similar results are found in the major studies performed in the field. Although our list of candidate predictors for mortality was more extensive than those in most prior studies, we did not identify new classes of risk factors. However, the way that we defined indicators of congestive heart failure (combination of shortness of breath and use of diuretics) contributes significantly to the discriminative power of the model.
Some programs considered the medical center in which the patient underwent operation as a composite indicator of process and found impressive differences among centers after adjustment for risk [7, 8, 15, 18]. Our study also found that operation in one of two outlier hospitals was an independent risk factor for early mortality, adjusting for case-mix variables.
The accuracy of the model has three dimensions. Resolution (measured as the area under the ROC curve [c statistic) tests the ability to discriminate between patients who have good or bad outcomes (living or dead). Calibration (goodness-of-fit) tests whether the model overestimates or underestimates the occurrence of the outcome over specified ranges of predicted outcomes. Cross-validation compares the area under the ROC curve between two independent populations in the same database.
The relative importance of these measures is dependent on the main purpose of the study. When the results are used to predict the outcome for individual patients, discrimination and cross-validation are of greater salience. However, if the model is used to determine an expected death rate for comparison with an actual death rate (eg, in an attempt to identify hospitals with higher-than-expected mortality, as in our case), then calibration may be more important. However, it should be noted that the logistic regression algorithm tends to optimize calibration rather than resolution.
Our models show high accuracy compared with other published models. The area under the ROC curve for our model, based on 4,835 patients, is 0.77 compared with 0.78 for the New York State model (based on 57,187 patients) [1]. Other models of CABGassociated mortality have reported areas under the ROC curve ranging from 0.74 to 0.76 [1618, 22]. However, formal statistical comparison between ROC areas under the curve requires an estimate of their precision (standard error); we are not in possession of these data for other models that specify the standard error of the area under the ROC curve.
Using our model for predicting mortality in individual patients, we can show that the probability of death for a 72-year-old woman with diabetes, mild congestive heart failure, and EF of 0.55, a creatinine level of 1.0 mg/100 mL, and three-vessel disease 30 days after an elective operation is 7.2%. The probability of death under the same circumstances for a 55-year-old man with no comorbidity, an EF of 0.50, a creatinine level of 1.2 mg/100 mL, and three-vessel disease is 0.9%.
It should be remembered that the accuracy for predicting mortality of the individual patient is limited. A trade-off exists between sensitivity and specificity. We prefer to increase the sensitivity, thus having fewer false-negative predictions. Therefore, we may find that the optimal point for our needs on the ROC curve differs from that reported (which assumes equal weighting for sensitivity and specificity).
The calibration of our models was also good. The p value of our case-mix model was 0.68 by the
2 test for the comparison of actual and predicted values, thus showing no significant difference between predicted and actual death rates. This value compares favorably with the goodness-of-fit statistic of the New York State model, for which the p value is 0.086 [1]. Moreover, using our model, the agreement of the predicted and actual death rates in the high-risk group is high, thereby verifying that the hospital comparisons did not undervalue or discriminate against centers that operated on patients with more severe disease.
The comparison between our model and other models shows that it may not be appropriate to apply a model created in one setting to other medical communities, even if agreement on variable definition is achieved. For example, the reoperation rate in the New York CABG registry is 9.5%, and prior CABG is an important risk factor in that population [7]. In our database, only 3.6% of operations are repeat procedures, and prior CABG is not a risk factor.
In only two studies [1, 18] has chronic lung disease been found to be a risk factor. Our prior hypothesis was that the definition of chronic lung disease may have been too vague, and therefore many smokers without major pulmonary disease may have been included in the chronic lung disease category. Therefore, we defined chronic lung disease on the basis of use of bronchodilators and steroids, but it still did not emerge as a risk factor. This may be due to patient selection, as few CABG procedures are performed in patients with moderate-to-severe chronic lung disease.
The comparison among hospitals shows that most medical centers operating in Israel (12/14) have a similar case-mix and nonsignificant differences between observed and actual mortality rates. This may indicate that in a relatively small medical community, the level of performance, even of a complicated procedure such as CABG, can be standardized.
In conclusion, a fair comparison among medical centers should include a local effort to create a risk model. This is necessary to produce a reasonably accurate risk stratification of patients and to make comparisons among institutions fair.
| Footnotes |
|---|
|
|
|---|
| Appendix 1 |
|---|
|
|
|---|
| Appendix 2 |
|---|
|
|
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
a In the event of discrepancies among sources of information, the positive response was accepted.
b Data were obtained at patient interview by trained nurses.
c Data were from chart or discharge report abstract.
d Detailed definitions of variables are as follows: Diabetes mellitus: patient treated with oral or insulin treatment; patients treated through diet only were not considered diabetic. Chronic lung disease; patient receiving regular therapy with ß-agonists or steroids; patients not receiving regular medical therapy were not considered to have chronic lung disease. Creatinine level: most recent value prior to operation. History of MI: MI prior to current hospitalization. Failed angioplasty: operation after acute occlusion of coronary artery after angioplasty during current hospitalization. Angina pectoris: typical = stabbing or pressue pain, parasternal or radiating to axilla, neck or jaw, or pain relieved by rest or nitroglycerin; atypical = any chest pain not included in aforementioned definitions. Angina pectoris severity (Canadian scale): 1 = agina occuring only with severe effort, 2 = angina ocurring with moderate effort, 3 = angina occurring with mild effort (less than 330 feet on level surface or less than one flight of stairs), and 4 = angina occurring with any physical effort and at rest. Clinical CHF: mild to moderate = on regimen of diuretic therapy, and no shortness of breath; severe = on regimen of diuretic therapy and reporting shortness of breath. Priority of operation: Urgent = operation performed less than 24 hours after nonelective admission to hospital; emergent = operation performed during immediate life-threatening conditions or resuscitation. Operation during acute MI: operation performed during first 24 hours of acute MI. Extent of diseased vessels: left main disease; obstruction of 50% or more of lumen of artery; other-vessel disease; obstruction of 70% or more of lumen of right coronary artery, circumflex artery, or left anterior descending coronary artery. CABG = coronary artery bypass grafting; CHF = congestive heart failure; IABP = intraaortic balloon pump; MI = myocardial infarction.
and Vladimir Yakirevitch, MD.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
R. Z. Omar, G. Ambler, P. Royston, J. Eliahoo, and K. M. Taylor Cardiac surgery risk modeling for mortality: a review of current practice and suggestions for improvement Ann. Thorac. Surg., June 1, 2004; 77(6): 2232 - 2237. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Galai, A. Israeli, Y. Zitser-Gurevich, and E. Simchen Is discharge policy a balanced decision between clinical considerations and hospital ownership policy? The CABG example J. Thorac. Cardiovasc. Surg., October 1, 2003; 126(4): 1018 - 1025. [Abstract] [Full Text] [PDF] |
||||
![]() |
V. A. Ferraris and S. P. Ferraris Risk Stratification and Comorbidity Card. Surg. Adult, January 1, 2003; 2(2003): 187 - 224. [Full Text] |
||||
![]() |
N. Sadeghi, S. Sadeghi, Z. A. Mood, and A. Karimi Determinants of operative mortality following primary coronary artery bypass surgery Eur. J. Cardiothorac. Surg., February 1, 2002; 21(2): 187 - 192. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. L. Grunkemeier, K. J. Zerr, and R. Jin Cardiac surgery report cards: making the grade Ann. Thorac. Surg., December 1, 2001; 72(6): 1845 - 1848. [Full Text] [PDF] |
||||
![]() |
E. Simchen, N. Galai, D. Braun, Y. Zitser-Gurevich, E. Shabtai, and I. Naveh Sociodemographic and clinical factors associated with low quality of life one year after coronary bypass operations: The Israeli Coronary Artery Bypass Study (ISCAB) J. Thorac. Cardiovasc. Surg., May 1, 2001; 121(5): 909 - 919. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y. Maor, Y. Cohen, L. Olmer, and B. Mozes Factors Associated With Health Indicators in Patients Undergoing Coronary Artery Bypass Surgery* Chest, December 1, 1999; 116(6): 1570 - 1574. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |