|
|
||||||||
Ann Thorac Surg 2005;80:2114-2119
© 2005 The Society of Thoracic Surgeons
a Division of Cardiothoracic Surgery, Oregon Health and Science University, Portland, Oregon
b Center for Research in the Implementation of Innovative Strategies in Practice (CRIISP), Iowa City VA Medical Center, Iowa City, Iowa
c Department of Internal Medicine, Division of General Internal Medicine, University of Iowa Carver College of Medicine, Iowa City, Iowa
Accepted for publication May 10, 2005.
* Address correspondence to Dr Welke, Division of Cardiothoracic Surgery L353, Oregon Health and Science University, 3181 SW Sam Jackson Park Rd, Portland, OR 97239-3098 (Email: welkek{at}ohsu.edu).
| Abstract |
|---|
|
|
|---|
METHODS: A retrospective cohort of 948,093 Medicare patients undergoing CABG in 870 US hospitals from 1996 to 2001 was categorized into quintiles, based on hospital CABG volume. Hospitals were also classified by volume criterion proposed by the Leapfrog Group. Logistic regression was used to adjust hospital mortality rates (in-hospital or within 30 days after CABG) for patient characteristics; discrimination of the volume categories was assessed by the c statistic.
RESULTS: The range in risk-adjusted mortality for hospitals within the quintiles was substantial: 1% to 17% at very low, 2% to 12% at low, 2% to 10% at medium, 2% to 9% at high, and 3% to 11% at very high volume hospitals. Moreover, volume alone was a poor discriminator of mortality (c statistic = 0.52). Similar variation in adjusted mortality was seen within the Leapfrog low-volume (1% to 17%) and high-volume groups (2% to 11%), and the Leapfrog criterion was a poor discriminator of mortality (c statistic = 0.51). Of the 660 low-volume Leapfrog hospitals, 253 (38%) had risk-adjusted mortality rates that were similar to or lower than the overall risk-adjusted mortality of high-volume hospitals (5.2%).
CONCLUSIONS: Volume alone, as a discriminator of mortality, is only slightly better than a coin flip (c statistic of 0.50).
| Introduction |
|---|
|
|
|---|
Much about the volume outcomes relationship remains unexamined, however. While nearly all prior studies of CABG have found statistically significant relationships between volume and mortality, these studies have typically utilized large health care databases and had exceptional statistical power to detect small differences across different categories of volume. Moreover, few studies have clearly articulated the strength of volume as a predictor of mortality and how patients should weigh it when choosing a hospital. Since patients are being encouraged to make their choice of hospital in part based on surgical volume, such information is essential, particularly for patients who may need to travel considerable distances to higher volume hospitals and leave their local support systems.
The purpose of this study is to quantify the strength of hospital volume as a predictor of an individual patient's risk of mortality after CABG, both in absolute terms and relative to the strength of previously validated approaches of risk-adjusting CABG hospital mortality rates. Such approaches may help illuminate the potential utility of volume as a marker of hospital quality, particularly for large employers and health plan administrators who have been encouraged to use volume as a criterion in selective contracting.
| Patients and Methods |
|---|
|
|
|---|
The eligible sample included 1,073,956 patients 65 years of age and older who underwent CABG in 1,174 US hospitals during a 6-year period from January 1, 1996, to December 31, 2001. Patients were identified on the basis of appropriate International Classification of Diseases, 9th Clinical Modification (ICD-9-CM) codes for CABG (36.1036.19). Consistent with prior studies [3, 10], patients who underwent cardiac valve replacement at the time of CABG (n = 70,242, 6.5%) were excluded from analysis, as were 318 patients (0.03%) discharged from hospitals with 5 or fewer CABG procedures in the MEDPAR data because of the possibility that such cases could be ICD-9 CM coding errors rather than actual CABG procedures. Lastly, 55,621 patients (5.1%) who underwent CABG at 225 hospitals that did not perform CABG during all 6 years of the study period were excluded to minimize the impact of random variation in mortality rates and maximize the robustness of volume outcome relationships. These exclusions left a final sample of 948,093 patients who underwent CABG in 870 US hospitals.
Hospital volume was defined as the average number of CABG procedures performed per year in Medicare beneficiaries over the 6 years of the study and was analyzed using four approaches. First, volume was evaluated as a continuous variable. Second, hospital volume was categorized into quintiles by selecting whole-number division points that most closely sorted the sample into five equal-size patient cohorts: very low (fewer than 125 yearly), low (125 to 204 yearly), medium (205 to 299 yearly), high (300 to 449 yearly), and very high (more than 449 yearly). Third, a similar approach was used to apportion the sample into deciles. Fourth, hospitals were placed into high- and low-volume categories using the current criterion of the Leapfrog Group of 450 procedures per year. Because the Leapfrog criterion considers total volume, a cutpoint of 256 was used for the current sample, given that roughly 57% of CABG procedures in the United States are performed in Medicare beneficiaries [3].
The primary study endpoint was mortality that occurred within 30 days of surgery or during the index hospitalization for CABG. Secondary endpoints included mortality that occurred within 7, 14, 30, 90, and 365 days of surgery.
Several approaches were used to define the relationships between hospital volume and mortality. First, unadjusted mortality rates across volume groups were compared using the chi-square statistic for linear trend. Second, the discrimination of volume alone as a predictor of mortality was assessed by the c statistic (11, 12) as determined from univariate logistic regression analyses for each of the four approaches for analyzing hospital volume described above. The c statistic is numerically equivalent to the area under a receiver operator characteristic curve and represents the proportion of times that a given patient who died had a higher predicted risk of death (on the basis of hospital volume in this case) than a given patient who lived. The c statistic ranges from 0.5 to 1.0 and has been widely reported for multivariable models predicting mortality for patients undergoing CABG. A test with perfect discrimination has a c statistic of 1.0. A test with no discrimination has a c statistic of 0.5, namely, a coin flip. To maximize potential discrimination, the quintile and decile volume categorizations were modeled using indicator variables for individual volume categories, in lieu of a single ordinal variable. The c statistics were also determined for the primary (30-day or in-hospital mortality) and each of the secondary endpoints (7-, 14-, 30-, 90-, and 365-day mortality) using the four approaches for modeling hospital volume.
Similar analyses were conducted for demographic and clinical variables in the MEDPAR data that were independently (p < 0.01) related to mortality and for other hospital characteristics, including teaching status (membership in the Council of Teaching Hospitals) and type of ownership (government, for profit, and nonprofit).
Multivariable analyses examined the increase in model discrimination (ie, c statistic values) resulting from the addition of hospital volume to models including patient-level independent predictors. Demographic variables included in the multivariable analyses included age, sex, and race. Clinical predictors included prior CABG surgery, surgical priority (elective, urgent, or emergent), comorbid conditions, and other potential markers of severity (eg, CABG performed on the same day as a cardiac catheterization or coronary angioplasty, primary diagnosis of acute myocardial infarction). In these analyses, age was represented as a categorical variable (65 to 69 years, 70 to 74, 75 to 79, 80 to 84, and 85 years and older), race was categorized as white, black, other, or missing, and comorbid conditions were defined on the basis of previously described algorithms for mapping ICD-9 CM codes to specific conditions [13, 14]. Models were estimated using generalized estimating equations to account for patients clustered within hospitals [15] and robust methods for calculating the 95% confidence intervals for regression coefficients [16]. A final set of analyses examined the range in risk-adjusted mortality rates for individual hospitals within individual volume categories. Risk-adjusted hospital mortality rates (30 day or in-hospital) were determined by multiplying the ratio of the observed to predicted mortality rate for an individual hospital by overall mortality rate in the entire sample (5.4%) [17]. Predicted rates for each hospital were determined by aggregating the predicted risks in individual patients, as estimated by the multivariable model of patient-level demographic and clinical variables independently related to mortality. Risk-adjusted mortality rates within individual categories of hospital volume (6-year average) were determined in a similar manner. All analyses were conducted using SAS statistical software (Version 8.1; SAS Institute, Cary, North Carolina).
| Results |
|---|
|
|
|---|
|
|
|
|
| Comment |
|---|
|
|
|---|
Our findings also highlight the peril in making clinical decisions (ie, selection of a hospital) solely on the basis of statistical considerations without considering the practical significance or magnitude of an effect size. This is particularly true for analyses using large health care databases, which have high power and the ability to detect very small effect sizes. In contrast to the use of volume as a criterion, a number of methods have been proposed for risk-adjusting CABG mortality data and for identifying higher performing hospitals. The c statistics for these approaches have ranged from 0.72 to 0.81 (Table 4), and thus explain substantially more of the variability in mortality than volume alone.
|
Our findings suggest that the effect of volume should be interpreted differently, particularly from the perspective of the patient, than has been previously suggested. Because the difference in mortality rates across volume categories is modest and the range of mortality rates for hospitals of similar volumes large, the impact of hospital volume on the mortality risk of an individual patient may be small. For example, the difference in risk-adjusted mortality of low- and high-volume hospitals, based on the Leapfrog criterion was 0.9%, indicating that roughly 110 patients would have to undergo CABG in high-volume hospitals to save one life 30 days after surgery. Such numbers are even higher if longer-term outcomes (eg, 90 or 365 days) are considered.
Patients choosing a hospital at which to undergo nonemergent or nonurgent cardiac surgery may seek counsel from a variety of sources. Recommendations from Internet consumer sites [8, 9] and the media [27, 28] that they seek a high-volume hospital may be in conflict with their wish to seek care at a more convenient or more comfortable lower volume hospital. In addition, they might not understand that their own risk characteristics and level of disease burden account for the vast majority of their mortality risk nor that any potential benefit from surgery at a higher volume institution may depend on their individual risk [29]. Thus, patients should be provided additional information about the actual benefits of undergoing surgery in a higher volume facility and the substantial likelihood that a lower volume hospital may have similar or lower mortality rates than a high-volume hospital.
Our analysis has several limitations. First, since we studied only Medicare patients, our findings may not be generalizable to patients less than 65 years of age. However, there is no evidence that age affects the volume outcome relationship. Second, our volumes are based on the Medicare population only. Although hospital Medicare volumes and total volumes are highly correlated, there may remain some degree of misclassification in our volume groupings as the proportion of Medicare patients in individual hospitals likely varies. Similarly, our categorizations of hospital volume may have been affected by differences in the proportions of patients enrolled in Medicare managed care plans. Claims data for such patients are typically excluded from the Medicare files that were used in our analysis. Third, we examined only hospital volumes and did not include the influence of surgeon volumes. Lastly, since we used administrative data, we may not have adequately accounted for differences in case mix and severity of illness. Thus, it is possible that volume would have had a larger effect on mortality rates that were adjusted using a more robust set of clinical variables that are found in administrative data. However, prior research suggests that hospital mortality rates that are adjusted using administrative and clinical data sets may be relatively similar [30].
In sum, the current study indicates that hospital volume alone is a poor discriminator of mortality and only marginally better than a coin flip. While many factors contribute to the mortality risk of a patient undergoing CABG, hospital volume is a relatively weak predictor. Moreover, there is a substantial likelihood that a lower volume hospital may have a mortality rate that is equal to that of a higher volume hospital. These findings highlight the need for the greater availability of hospital-specific performance data (eg, risk-adjusted mortality rates, process of care measures that lead to better outcomes), so that patients are not encouraged to select hospitals solely on the basis of volume.[11, 12]
| Acknowledgments |
|---|
|
|
|---|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
D. M. Shahian, S. M. O'Brien, S.-L. T. Normand, E. D. Peterson, and F. H. Edwards Association of hospital coronary artery bypass volume with processes of care, mortality, morbidity, and the Society of Thoracic Surgeons composite quality score J. Thorac. Cardiovasc. Surg., February 1, 2010; 139(2): 273 - 282. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. P. Kernisan, S. J. Lee, W. J. Boscardin, C. S. Landefeld, and R. A. Dudley Association Between Hospital-Reported Leapfrog Safe Practices Scores and Inpatient Mortality JAMA, April 1, 2009; 301(13): 1341 - 1348. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. K. Song, B. S. Diggs, M. S. Slater, S. W. Guyton, R. M. Ungerleider, and K. F. Welke Improved quality and cost-effectiveness of coronary artery bypass grafting in the United States from 1988 to 2005. J. Thorac. Cardiovasc. Surg., January 1, 2009; 137(1): 65 - 69. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. M. Shahian and S.-L. T. Normand Low-volume coronary artery bypass surgery: Measuring and optimizing performance. J. Thorac. Cardiovasc. Surg., June 1, 2008; 135(6): 1202 - 1209. [Full Text] [PDF] |
||||
![]() |
D. M. Holzhey, S. Jacobs, T. Walther, M. Mochalski, F. W. Mohr, and V. Falk Cumulative sum failure analysis for eight surgeons performing minimally invasive direct coronary artery bypass J. Thorac. Cardiovasc. Surg., September 1, 2007; 134(3): 663 - 669. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Rodgers, B. A. Jobe, R. W. O'Rourke, B. Sheppard, B. Diggs, and J. G. Hunter Case Volume as a Predictor of Inpatient Mortality After Esophagectomy Arch Surg, September 1, 2007; 142(9): 829 - 839. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. Gauvreau Reevaluation of the Volume-Outcome Relationship for Pediatric Cardiac Surgery Circulation, May 22, 2007; 115(20): 2599 - 2601. [Full Text] [PDF] |
||||
![]() |
E. Losina, E. A. Wright, C. L. Kessler, J. A. Barrett, A. H. Fossel, A. H. Creel, N. N. Mahomed, J. A. Baron, and J. N. Katz Neighborhoods Matter: Use of Hospitals With Worse Outcomes Following Total Knee Replacement by Patients From Vulnerable Populations Arch Intern Med, January 22, 2007; 167(2): 182 - 186. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. A. Herbert Invited commentary Ann. Thorac. Surg., December 1, 2005; 80(6): 2119 - 2120. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |