Ann Thorac Surg 2008;86:1405-1408. doi:10.1016/j.athoracsur.2008.07.027
© 2008 The Society of Thoracic Surgeons
Editorial
Measuring the Quality of Surgical Outcomes in General Thoracic Surgery: Should Surgical Volume Be Used to Direct Patient Referrals?
Benjamin D. Kozower, MDa,*,
George J. Stukenborg, PhDb,
Christine L. Lau, MDa,
David R. Jones, MDa
a Department of Surgery, University of Virginia Health System, Charlottesville, Virginia
b Department of Public Health Sciences, University of Virginia Health System, Charlottesville, Virginia
* Address correspondence to Dr Kozower, University of Virginia Health System, General Thoracic Surgery, PO Box 800679, Charlottesville, VA 22908-0679 (Email: bdk8g{at}virginia.edu).
 |
Introduction
|
|---|
Surgeons and hospitals face an increasing demand to provide evidence for the quality of care they deliver. However, considerable debate exists regarding which outcomes measures to use to reflect surgical quality [1]. Structural measures are a broad group of variables that reflect the setting in which care is delivered [2]. Volume is one important and easily quantifiable structural measure. Many studies have demonstrated an inverse relationship between the volume of hospital surgical procedures and postoperative mortality. These studies have been used to recommend the regionalization of surgical procedures using selected volume thresholds. However, the volume-outcome relationship is extremely complex, and debate continues as to how it should be used by public and private organizations caring for their patients. This policy analysis examines the evidence regarding volume as a quality indicator for general thoracic surgery and outlines potential unintended consequences of using volume to direct patient referrals.
|
Support for the Volume-Outcome Relationship
|
|---|
The seminal article describing the relationship between increasing case volume and improved outcomes was published by Luft and colleagues [3] in 1979. They demonstrated that hospitals where certain complicated operations were performed 200 or more times annually had case-adjusted death rates up to 41% lower than hospitals with lower volumes. Numerous articles have supported this volume-outcome hypothesis and Birkmeyer and colleagues [4–6] have published extensive articles on this subject.
Administrative data has been used to illustrate that hospital volume is inversely related to outcome [4]. However, little was known about the possible mechanisms underlying these observed associations. Because high-volume centers tend to be larger facilities, they may have a broader range of specialists, improved technology-based services, better-staffed intensive care units, and other resources not available at smaller centers. Thus, by virtue of their size, high-volume centers may be better equipped to deliver care for high-risk surgical procedures [4]. Several other studies have also used administrative databases to demonstrate that mortality risk after lung cancer resection is significantly lower in hospitals with high volumes [7–9]. These studies showed a decrease of 1% to 4% in 30-day mortality rates between the highest and lowest volume centers.
Birkmeyer and colleagues [5] have also demonstrated that surgeon volume is directly related to outcome. Using information from the national Medicare claims database, they examined mortality among 474,108 patients who underwent one of eight cardiovascular procedures or cancer resections. They demonstrated that surgeon volume was inversely related to operative mortality for all eight high-risk operative procedures, including lung and esophageal cancer resection. The adjusted odds ratios for postoperative mortality for low-volume surgeons compared with high-volume surgeons were 1.24 for lung cancer resection and 2.30 for esophageal cancer resection. In addition, Birkmeyer and colleagues [5] demonstrated that much of the observed association between hospital volume and operative mortality was mediated by individual surgeon volume. The authors concluded that patients may be able to improve their chances of survival substantially, even at high-volume hospitals, by selecting surgeons who perform particular operations frequently.
|
Evidence Refuting the Volume-Outcome Relationship
|
|---|
In 1997, the Veterans Affairs (VA) administration considered closing low-volume centers to decrease cost and improve outcome [10]. To determine the feasibility of this plan, the National Surgical Quality Improvement Program (NSQIP) examined the relationship of surgical volume to outcome in eight common operations [11]. Four different types of statistical analyses showed no relationship between the 30-day mortality observed to expected ratio and procedure volume in any of the eight operations examined. In fact, in this study, the hospital with the highest volume of colectomies was one of three outliers in the 22-hospital system, and a hospital in the lower quartile for volume had the lowest mortality. In the face of this compelling data, managers in the VA system did not institute the volume-based system.
Why did this VA study contradict the intuitive findings of other volume-outcome studies? The use of clinical rather than administrative data in previous studies supporting the volume-outcome relationship is crucial because administrative data have limited ability to differentiate comorbidity, severity of illness, and case mix [12–14]. This may contribute to selection bias in studies using administrative data as patient risk factors were not accounted for in many earlier studies. The VA study used a risk-adjusted clinical database to compare outcomes. The NSQIP also developed a system to accurately track outcomes, distribute the information to all VA centers, identify important process issues, and implement change [10, 15]. The primary focus of the NSQIP program was to improve quality, and this program resulted in a 27% reduction in mortality and a 45% reduction in morbidity. The NSQIP data is independently verified for accuracy and it has been underscored that quality comes from excellent systems of care, regardless of volume. The NSQIP program provided a direct outcome-based measure of quality and eliminated the need to use volume of surgery as a proxy measure of quality.
In an effort to improve the risk-adjustment of administrative data, Stukenborg and colleagues [16] used present-at-admission diagnoses. This technique improved the performance characteristics of their model compared with other risk adjustment strategies using administrative data, which do not take the present-at-admission diagnoses into account. They evaluated the volume-outcome relationship in lung cancer using 14,456 California hospital patients. Importantly, large increases in procedure volume were associated with much smaller decreases in mortality risk when compared with other risk-adjustment models. They also found that the volume of lung cancer operations during the year prior to a patient's admission was not a statistically significant predictor of mortality after lung cancer resection.
Other studies have questioned the relationship between volume and outcome in thoracic surgery. Treasure and colleagues [17] used clinical data from the United Kingdom Society of Cardiothoracic Surgeons to evaluate 4,028 lung cancer resections between 1999 and 2001. They found that the number of procedures performed by a particular thoracic surgeon was not related to in-hospital mortality. The Spanish Cooperative group investigated the relationship between hospital volume, mortality, and complications after lung cancer surgery [18]. They grouped hospitals into terciles by volume and also found no significant volume-outcome relationship for mortality.
Rodgers and colleagues [19] hypothesized that volume was a poor predictor of mortality after esophagectomy. The purpose of their study was to quantify the factors that influence mortality and to analyze the volume-outcome relationship. They found a complex relationship between volume and outcome when looking at individual surgeons and hospitals, with a wide scatter in the strength of the relationship. Their primary conclusion was that patient factors have a greater influence on inpatient mortality than volume. Importantly, using volume criteria alone to choose a provider could actually increase the risk of mortality for certain patients.
|
Policy Initiatives Based on the Volume-Outcome Relationship
|
|---|
A natural consequence of information on the volume-outcome relationship has been policy aimed at directing patients to high-volume centers. This has been most systematically applied by the Leapfrog Group, which is an initiative driven by organizations that buy healthcare "who are working to improve the safety, quality, and affordability of healthcare" (http://www.leapfroggroup.org). Leapfrog consists of more than 150 healthcare purchasers and currently provides health benefits to more than 37 million Americans in all 50 states. They advocate that high-risk procedures, such as an esophagectomy, should be performed only at institutions with certain annual caseload volumes [20]. Some estimates report that as many as 65,000 lives could be saved annually if the Leapfrog initiatives were applied nationally [21].
Shukri F. Khuri, a surgeon and lead investigator in NSQIP, has written extensively on the volume-outcome relationship. He has challenged parts of the Leapfrog initiative and its claim that a referral strategy based on the volume-outcome relationship will save lives [15, 22]. To estimate the numbers reported by Birkmeyer and colleagues [4, 5], 10 different assumptions were made, each with an inherent potential for error. Even if each of the assumptions is permissible on its own, the aggregate of the potential errors in the combined assumptions compromises the validity of the primary endpoint (ie, lives saved). Identifying an actual volume threshold value is also very difficult. The majority of reports supporting the volume-outcome relationship have used arbitrary partitioning, and it is unknown whether those specific values are better than their alternatives [4, 5].
If one works under the assumption that volume-based referral strategies will save lives, whether or not they will reduce costs remains unclear. There has been a recent consumer-oriented push to improve the distribution of volume information to patients (www.healthscope.org) [22]. In addition, individual states are incorporating volume standards in certificate-of-need applications for new surgical centers. However, the economic implications are poorly understood and will likely vary according to the perspective of the participant. For example, volume-based referral strategies will redistribute surgical revenues to high-volume centers, improving profits for these high-volume centers, providing that the costs of delivering care remain lower than the average reimbursements. Low-volume centers will likely see significant decreases in revenue if volume-based strategies are used. Interestingly, both government and private insurers may end up paying more for procedures in a volume-based referral system. If high-volume centers improve their market share and gain negotiating power, the decrease in competition may enable them to demand increased payments. The true effects of volume-based referrals will depend on the details of the system used. However, it is crucial that policy makers understand that this kind of system is not a guarantee for lower costs.
|
Practical Considerations for the Volume-Outcome Relationship
|
|---|
A major issue with the use of volume as an indicator of quality is the problem of sample size. The Agency for Healthcare Research and Quality advocated that mortality should be an indicator of quality for seven major operations [23]. However, many of these procedures are performed infrequently at individual hospitals, and the mortality rates for a given procedure can have large variations. Therefore, it may be very difficult to identify a difference in mortality rates based on volume. Dimick and colleagues [24] evaluated this issue by looking at the proportion of hospitals in the United States that perform enough of these seven high-risk procedures to accurately detect a doubling in the mortality rate. With the exception of coronary artery bypass grafting, they concluded that the procedures for which surgical mortality has been advocated as a quality indicator are not performed frequently enough to detect a doubling in the mortality rate.
Several unintended consequences of volume-based referral strategies have not received thorough investigation. First, such strategies could have a devastating financial impact on smaller, lower-volume hospitals, even though such hospitals may provide high-quality care. Second, some patients could be forced to travel long distances to obtain planned care. An examination of Medicare data demonstrated that more than half of patients undergoing procedures in low-volume hospitals lived in regions lacking a high-volume center [25]. Some regions may not have a large enough population to support a high-volume center and others may have many centers, each performing low volumes. Birkmeyer and colleagues [26] demonstrated that if the volume thresholds are set relatively low (two per year for an esophagectomy), approximately 15% of patients would change to higher-volume centers, which are still within a reasonable travel area. However, with high-volume standards (> 19 per year for esophagectomy), approximately 80% of patients would change to higher-volume centers with significant increases in travel time. Higher volume centers also may not have the physical capacity to absorb more patients, requiring significant changes to hospital infrastructure. Another important issue is that a volume-based referral strategy could lead to the closing of low-volume training programs. Will high-volume hospitals have the resources to adequately train additional people? The worst potential consequence of regionalization of volume-based referral strategies would be the absence of care for patients who may not be able to go to a regional center for financial, logistical, or personal reasons. Finally, regardless of whether or not volume is a good proxy measurement for quality, it will never be perfect. Some low-volume centers will be able to document excellent surgical results, whereas some high-volume centers will have poorer outcomes.
|
Future Measures of Surgical Quality
|
|---|
Surgeons would argue that they, and not the insurance industry, should set standards for surgical care. A crucial step will be surgeon participation in the development of accurate risk-adjusted data. The NSQIP program was an excellent start, but a tremendous difference exists between the VA system and the rest of American surgery. The VA was able to insist on mandatory reporting and designed a risk-adjustment system. The government pays to train the data coordinators and for independent audits. They have a relatively good electronic medical record that helps with data collection. The VA centers have protected communication between them, and the VA has created an environment in which the data was used to improve quality rather than penalize poor performers. The NSQIP program is indeed an excellent example of surgeons directing the quality of surgical care.
In an effort for general thoracic surgeons to direct quality assessment, The Society of Thoracic Surgeons General Thoracic Surgery Database was initiated in 2002. It currently contains over 30,000 patient records and 106 participating sites [27]. The Society of Thoracic Surgeons cardiac surgery database is the gold standard for clinical data analysis registries. It has been instrumental in determining risk-adjusted outcome measures and influencing public policy. The general thoracic surgery database is only in its infancy, but it continues to grow and to illustrate the understanding that surgeons have for setting their own benchmarks based on accurate, detailed, clinical data. The major impediments to its expansion have been the administrative costs for participating surgeons and the time required for data collection and reporting.
Another important predictor of quality is the level of surgeon training. Many articles have demonstrated that specialists have better outcomes for high-risk procedures than general surgeons [28, 29]. These results are independent of hospital and surgeon volume. This is a complicated issue for credentialing organizations. More stringent implementation of volume thresholds will further restrict privileges for certain procedures and have significant impact on smaller regional hospitals, general surgeons with diverse practices, and the delivery of emergency care.
Finally, are thoracic surgeons and The Society of Thoracic Surgeons database adequately measuring quality? Surgeons have done a remarkable job recording and improving on their 30-day outcomes; however, keeping track of patients for longer time periods and incorporating meaningful measures of functional outcomes and quality of life have been more difficult. The current data points in surgical registries do not reflect patient and family satisfaction or the quality of life after procedures. This enormously important information continues to lag behind other quality improvement initiatives. It is imperative that the thoracic surgery community incorporates these outcome measures into an accurate clinical database that will enable physicians and patients to make informed decisions.
Further research is required to determine whether volume-based referral strategies will improve the quality of surgical care in the United States. The volume-outcome relationship is extremely complicated and the financial and ethical implications of a volume-based referral strategy are equally complex. The inverse volume-outcome relationship that was demonstrated using administrative data has blurred with the use of accurate clinical data and more accurate methods of risk-adjusting administrative data. It is imperative that the strength of the volume-outcome relationship be re-evaluated as better data becomes available. It is equally important that we better understand the economic and feasibility implications of such a system.
|
References
|
|---|
- Birkmeyer JD, Dimick JB, Birkmeyer NJ. Measuring the quality of surgical care: structure, process, or outcomes? J Am Coll Surg 2004;198:626-632.[Medline]
- Birkmeyer NJ, Birkmeyer JD. Strategies for improving surgical quality—should payers reward excellence or effort? N Engl J Med 2006;354:864-870.[Medline]
- Luft HS, Bunker JP, Enthoven AC. Should operations be regionalized?. The empirical relation between surgical volume and mortality. N Engl J Med 1979;301:1364-1369.[Medline]
- Birkmeyer JD, Siewers AE, Finlayson EV, et al. Hospital volume and surgical mortality in the united states N Engl J Med 2002;346:1128-1137.[Medline]
- Birkmeyer JD, Stukel TA, Siewers AE, Goodney PP, Wennberg DE, Lucas FL. Surgeon volume and operative mortality in the united states N Engl J Med 2003;349:2117-2127.[Medline]
- Birkmeyer JD, Sun Y, Wong SL, Stukel TA. Hospital volume and late survival after cancer surgery Ann Surg 2007;245:777-783.[Medline]
- Bach PB, Cramer LD, Schrag D, Downey RJ, Gelfand SE, Begg CB. The influence of hospital volume on survival after resection for lung cancer N Engl J Med 2001;345:181-188.[Medline]
- Romano PS, Mark DH. Patient and hospital characteristics related to in-hospital mortality after lung cancer resection Chest 1992;101:1332-1337.[Medline]
- Finlayson EV, Goodney PP, Birkmeyer JD. Hospital volume and operative mortality in cancer surgery: A national study Arch Surg 2003;138:721-725.[Abstract/Free Full Text]
- Khuri SF. Quality, advocacy, healthcare policy, and the surgeon Ann Thorac Surg 2002;74:641-649.[Free Full Text]
- Khuri SF, Daley J, Henderson W, et al. Relation of surgical volume to outcome in eight common operations: Results from the VA national surgical quality improvement program Ann Surg 1999;230:414-429.[Medline]
- Hsia DC. Accuracy of medicare reimbursement for cardiac arrest JAMA 1990;264:59-62.[Abstract/Free Full Text]
- Fisher ES, Whaley FS, Krushat WM, et al. The accuracy of medicare's hospital claims data: Progress has been made, but problems remain Am J Public Health 1992;82:243-248.[Medline]
- Iezzoni LI, Foley SM, Daley J, Hughes J, Fisher ES, Heeren T. Comorbidities, complications, and coding bias. Does the number of diagnosis codes matter in predicting in-hospital mortality?. JAMA 1992;267:2197-2203.[Abstract/Free Full Text]
- Khuri SF. Invited commentary: surgeons, not general motors, should set standards for surgical care Surgery 2001;130:429-431.[Medline]
- Stukenborg GJ, Kilbridge KL, Wagner DP, et al. Present-at-admission diagnoses improve mortality risk adjustment and allow more accurate assessment of the relationship between volume of lung cancer operations and mortality risk Surgery 2005;138:498-507.[Medline]
- Treasure T, Utley M, Bailey A. Assessment of whether in-hospital mortality for lobectomy is a useful standard for the quality of lung cancer surgery: Retrospective study BMJ 2003;327:73.[Abstract/Free Full Text]
- Freixinet JL, Julia-Serda G, Rodriguez PM, et al. Hospital volume: operative morbidity, mortality and survival in thoracotomy for lung cancer. A Spanish multicenter study of 2994 cases. Eur J Cardio Surg 2006;29:20-25.
- Rodgers M, Jobe BA, O'Rourke RW, Sheppard B, Diggs B, Hunter JG. Case volume as a predictor of inpatient mortality after esophagectomy Arch Surg 2007;142:829-839.[Abstract/Free Full Text]
- Birkmeyer JD, Finlayson EV, Birkmeyer CM. Volume standards for high-risk surgical procedures: Potential benefits of the leapfrog initiative Surg 2001;130:415-422.
- Birkmeyer JD, Dimick JB. Potential benefits of the new leapfrog standards: Effect of process and outcomes measures Surg 2004;135:569-575.
- Birkmeyer JD, Skinner JS, Wennberg DE. Will volume-based referral strategies reduce costs or just save lives? Health Aff 2002;21:234-241.[Abstract/Free Full Text]
- AHRQ quality indicators—guide to inpatient quality indicators: quality of care in hospitals—volume, mortality, and utilization.
- Dimick JB, Welch HG, Birkmeyer JD. Surgical mortality as an indicator of hospital quality: the problem with small sample size JAMA 2004;292:847-851.[Abstract/Free Full Text]
- Dimick JB, Finlayson SR, Birkmeyer JD. Regional availability of high-volume hospitals for major surgery Health Aff 2004:AR45-AR53.
- Birkmeyer JD, Siewers AE, Marth NJ, Goodman DC. Regionalization of high-risk surgery and implications for patient travel times JAMA 2003;290:2703-2708.[Abstract/Free Full Text]
- Wright CD, Edwards FH, Society of Thoracic Surgeons General Thoracic Surgery Database Task Force, Society of Thoracic Surgeons Workforce on National Databases The Society of Thoracic Surgeons general thoracic surgery database Ann Thorac Surg 2007;83:893-894.[Free Full Text]
- Goodney PP, Lucas FL, Stukel TA, Birkmeyer JD. Surgeon specialty and operative mortality with lung resection Ann Surg 2005;241:179-184.[Medline]
- Dimick JB, Goodney PP, Orringer MB, Birkmeyer JD. Specialty training and mortality after esophageal cancer resection Ann Thorac Surg 2005;80:282-286.[Abstract/Free Full Text]
Top
Introduction
Support for the Volume-Outcome...