| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Ann Thorac Surg 2001;72:2155-2168
© 2001 The Society of Thoracic Surgeons
a Department of Thoracic and Cardiovascular Surgery, Lahey Clinic, Burlington, Massachusetts, USA
b Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts, USA
c Department of Biostatistics, Harvard School of Public Health, Boston, Massachusetts, USA
d Division of Cardiac Surgery, Massachusetts General Hospital, Boston, Massachusetts, USA
e Division of Cardiology, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
f Department of Cardiology, St. Elizabeth’s Medical Center, Boston, Massachusetts, USA
g Department of Cardiology, Brigham and Women’s Hospital and Cardiovascular Data Analysis Center, Boston, Massachusetts, USA
h Division of Health Care Quality, Massachusetts Department of Public Health, Boston, Massachusetts, USA
* Address reprint requests to Dr Shahian, Department of Thoracic and Cardiovascular Surgery, Lahey Clinic, 41 Mall Rd, Burlington, MA 01805, USA
e-mail: david.m.shahian{at}lahey.org
Public report cards and confidential, collaborative peer education represent distinctly different approaches to cardiac surgery quality assessment and improvement. This review discusses the controversies regarding their methodology and relative effectiveness.
Report cards have been the more commonly used approach, typically as a result of state legislation. They are based on the presumption that publication of outcomes effectively motivates providers, and that market forces will reward higher quality. Numerous studies have challenged the validity of these hypotheses. Furthermore, although states with report cards have reported significant decreases in risk-adjusted mortality, it is unclear whether this improvement resulted from public disclosure or, rather, from the development of internal quality programs by hospitals. An additional confounding factor is the nationwide decline in heart surgery mortality, including states without quality monitoring. Finally, report cards may engender negative behaviors such as high-risk case avoidance and "gaming" of the reporting system, especially if individual surgeon results are published.
The alternative approach, continuous quality improvement, may provide an opportunity to enhance performance and reduce interprovider variability while avoiding the unintended negative consequences of report cards. This collaborative method, which uses exchange visits between programs and determination of best practice, has been highly effective in northern New England and in the Veterans Affairs Administration. However, despite their potential advantages, quality programs based solely on confidential continuous quality improvement do not address the issue of public accountability. For this reason, some states may continue to mandate report cards. In such instances, it is imperative that appropriate statistical techniques and report formats are used, and that professional organizations simultaneously implement continuous quality improvement programs.
The statistical methodology underlying current report cards is flawed, and does not justify the degree of accuracy presented to the public. All existing risk-adjustment methods have substantial inherent imprecision, and this is compounded when the results of such patient-level models are aggregated and used inappropriately to assess provider performance. Specific problems include sample size differences, clustering of observations, multiple comparisons, and failure to account for the random component of interprovider variability. We advocate the use of hierarchical or multilevel statistical models to address these concerns, as well as report formats that emphasize the statistical uncertainty of the results.
This article has been cited by other articles:
|
|
 |
|
  E. S. Holmboe, Y. Wang, T. P. Meehan, J. P. Tate, S.-Y. Ho, K. S. Starkey, and R. S. Lipner Association Between Maintenance of Certification Examination Scores and Quality of Care for Medicare Beneficiaries Arch Intern Med, July 14, 2008; 168(13): 1396 - 1403. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. E. Drye and J. Chen Evaluating Quality in Small-Volume Hospitals Arch Intern Med, June 23, 2008; 168(12): 1249 - 1251. [Full Text] [PDF]
|
|
|
|
 |
|
 D. M. Shahian and S.-L. T. Normand Low-volume coronary artery bypass surgery: Measuring and optimizing performance. J. Thorac. Cardiovasc. Surg., June 1, 2008; 135(6): 1202 - 1209. [Full Text] [PDF]
|
|
|
|
 |
|
 D. M. Shahian and S.-L. T. Normand Comparison of "Risk-Adjusted" Hospital Outcomes Circulation, April 15, 2008; 117(15): 1955 - 1963. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. E. Landon and S.-L. T. Normand Performance Measurement in the Small Office Practice: Challenges and Potential Solutions Ann Intern Med, March 4, 2008; 148(5): 353 - 357. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 V. A. Ferraris, F. H. Edwards, D. M. Shahian, and S. P. Ferraris Risk Stratification and Comorbidity Card. Surg. Adult, January 1, 2008; 3(2008): 199 - 246. [Full Text]
|
|
|
|
 |
|
 E. H. Livingston, A. C. Elliott, L. S. Hynan, and E. Engel When Policy Meets Statistics: The Very Real Effect That Questionable Statistical Analysis Has on Limiting Health Care Access for Bariatric Surgery Arch Surg, October 1, 2007; 142(10): 979 - 987. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. Hudorovic ICVTS on-line discussion A EuroSCORE and specialised databases Interactive CardioVascular and Thoracic Surgery, August 1, 2007; 6(4): 473 - 473. [Full Text] [PDF]
|
|
|
|
|
|
L. G. Bazzani and J. P. Marcin Case Volume and Mortality in Pediatric Cardiac Surgery Patients in California, 1998-2003 Circulation, May 22, 2007; 115(20): 2652 - 2659. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. M. Shahian, F. H. Edwards, V. A. Ferraris, C. K. Haan, J. B. Rich, S.-L. T. Normand, E. R. DeLong, S. M. O'Brien, C. M. Shewan, R. S. Dokholyan, et al. Quality Measurement in Adult Cardiac Surgery: Part 1--Conceptual Framework and Measure Selection Ann. Thorac. Surg., April 1, 2007; 83(4_Supplement): S3 - S12. [Full Text] [PDF]
|
|
|
|
|
|
S. M. O'Brien, D. M. Shahian, E. R. DeLong, S.-L. T. Normand, F. H. Edwards, V. A. Ferraris, C. K. Haan, J. B. Rich, C. M. Shewan, R. S. Dokholyan, et al. Quality Measurement in Adult Cardiac Surgery: Part 2--Statistical Considerations in Composite Measure Scoring and Provider Rating Ann. Thorac. Surg., April 1, 2007; 83(4_Supplement): S13 - S26. [Full Text] [PDF]
|
|
|
|
|
|
D. M. Shahian, T. Silverstein, A. F. Lovett, R. E. Wolf, and S.-L. T. Normand Comparison of Clinical and Administrative Data Sources for Hospital Coronary Artery Bypass Graft Surgery Report Cards Circulation, March 27, 2007; 115(12): 1518 - 1527. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S.-L. T. Normand, R. E. Wolf, J. Z. Ayanian, and B. J. McNeil Assessing the Accuracy of Hospital Clinical Performance Measures Med Decis Making, January 1, 2007; 27(1): 9 - 20. [Abstract] [PDF]
|
|
|
|
 |
|
 A. J. Epstein Do cardiac surgery report cards reduce mortality? Assessing the evidence. Med Care Res Rev, August 1, 2006; 63(4): 403 - 426. [Abstract] [PDF]
|
|
|
|
|
|
J. Nilsson, M. Ohlsson, L. Thulin, P. Hoglund, S. A.M. Nashef, and J. Brandt Risk factor identification and mortality prediction in cardiac surgery using artificial neural networks J. Thorac. Cardiovasc. Surg., July 1, 2006; 132(1): 12 - 19. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. M. Krumholz, Y. Wang, J. A. Mattera, Y. Wang, L. F. Han, M. J. Ingber, S. Roman, and S.-L. T. Normand An Administrative Claims Model Suitable for Profiling Hospital Performance Based on 30-Day Mortality Rates Among Patients With an Acute Myocardial Infarction Circulation, April 4, 2006; 113(13): 1683 - 1692. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. M. Krumholz, Y. Wang, J. A. Mattera, Y. Wang, L. F. Han, M. J. Ingber, S. Roman, and S.-L. T. Normand An Administrative Claims Model Suitable for Profiling Hospital Performance Based on 30-Day Mortality Rates Among Patients With Heart Failure Circulation, April 4, 2006; 113(13): 1693 - 1701. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Fortuna, M. Vizioli, A. Contini, I. Ghidoni, B. Biagi, D. Pacini, G. Carnelos, and R. Grilli Assessing clinical performance in cardiac surgery. Does a specialised clinical database make a difference? Interactive CardioVascular and Thoracic Surgery, April 1, 2006; 5(2): 123 - 127. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. M. Krumholz, R. G. Brindis, J. E. Brush, D. J. Cohen, A. J. Epstein, K. Furie, G. Howard, E. D. Peterson, S. S. Rathore, S. C. Smith Jr, et al. Standards for Statistical Models Used for Public Reporting of Health Outcomes: An American Heart Association Scientific Statement From the Quality of Care and Outcomes Research Interdisciplinary Writing Group: Cosponsored by the Council on Epidemiology and Prevention and the Stroke Council Endorsed by the American College of Cardiology Foundation Circulation, January 24, 2006; 113(3): 456 - 462. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Shahian, D. F. Torchiana, R. J. Shemin, J. D. Rawn, and S.-L. T. Normand Massachusetts Cardiac Surgery Report Card: Implications of Statistical Methodology Ann. Thorac. Surg., December 1, 2005; 80(6): 2106 - 2113. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Shahian, D. F. Torchiana, and S.-L. T. Normand Implementation of a Cardiac Surgery Report Card: Lessons From the Massachusetts Experience Ann. Thorac. Surg., September 1, 2005; 80(3): 1146 - 1150. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Ambler, R. Z. Omar, P. Royston, R. Kinsman, B. E. Keogh, and K. M. Taylor Generic, Simple Risk Stratification Model for Heart Valve Surgery Circulation, July 12, 2005; 112(2): 224 - 231. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Z. G. Turi The Big Chill: The Deleterious Effects of Public Reporting on Access to Health Care for the Sickest Patients J. Am. Coll. Cardiol., June 7, 2005; 45(11): 1766 - 1768. [Full Text] [PDF]
|
|
|
|
|
|
M. J. Mack, M. Herbert, S. Prince, T. M. Dewey, M. J. Magee, and J. R. Edgerton Does reporting of coronary artery bypass grafting from administrative databases accurately reflect actual clinical outcomes? J. Thorac. Cardiovasc. Surg., June 1, 2005; 129(6): 1309 - 1317. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. J. Bold A Young Surgeon's Perspective Arch Surg, March 1, 2005; 140(3): 254 - 257. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Shahian, E. H. Blackstone, F. H. Edwards, F. L. Grover, G. L. Grunkemeier, D. C. Naftel, S. A.M. Nashef, W. C. Nugent, and E. D. Peterson Cardiac Surgery Risk Models: A Position Article Ann. Thorac. Surg., November 1, 2004; 78(5): 1868 - 1877. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. A M Nashef and R. J F Baskett Quality Assessment in Cardiac Surgery: Do Not Miss the Boat! Asian Cardiovasc Thorac Ann, September 1, 2004; 12(3): 187 - 189. [Full Text] [PDF]
|
|
|
|
 |
|
 B. Keogh, D. Spiegelhalter, A. Bailey, J. Roxburgh, P. Magee, and C. Hilton The legacy of Bristol: public disclosure of individual surgeons' results BMJ, August 21, 2004; 329(7463): 450 - 454. [Full Text] [PDF]
|
|
|
|
|
|
M. A. Herbert, S. L. Prince, J. L. Williams, M. J. Magee, and M. J. Mack Are unaudited records from an outcomes registry database accurate? Ann. Thorac. Surg., June 1, 2004; 77(6): 1960 - 1964. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Z. Omar, G. Ambler, P. Royston, J. Eliahoo, and K. M. Taylor Cardiac surgery risk modeling for mortality: a review of current practice and suggestions for improvement Ann. Thorac. Surg., June 1, 2004; 77(6): 2232 - 2237. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. M. Shahian Improving Cardiac Surgery Quality--Volume, Outcome, Process? JAMA, January 14, 2004; 291(2): 246 - 248. [Full Text] [PDF]
|
|
|
|
|
|
D. E. Cutlip, K. K. L. Ho, R. E. Kuntz, and D. S. Baim Risk assessment for percutaneous coronary intervention: our version of the weather report? J. Am. Coll. Cardiol., December 3, 2003; 42(11): 1896 - 1899. [Full Text] [PDF]
|
|
|
|
|
|
B. E. Landon, S.-L. T. Normand, D. Blumenthal, and J. Daley Physician Clinical Performance Assessment: Prospects and Barriers JAMA, September 3, 2003; 290(9): 1183 - 1189. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
I. S Gilfillan Ranking heart surgeons has pitfalls BMJ, July 10, 2003; 327(7406): 107 - 107. [Full Text]
|
|
|
|
|
|
R. P. Anderson, R. Jin, and G. L. Grunkemeier Understanding logistic regression analysis in clinical reports: an introduction Ann. Thorac. Surg., March 1, 2003; 75(3): 753 - 757. [Full Text] [PDF]
|
|
|
|
|
|
D. M. Shahian and S.-L. T. Normand The volume-outcome relationship: from Luft to Leapfrog Ann. Thorac. Surg., March 1, 2003; 75(3): 1048 - 1058. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. A. Ferraris and S. P. Ferraris Risk Stratification and Comorbidity Card. Surg. Adult, January 1, 2003; 2(2003): 187 - 224. [Full Text]
|
|
|
|
|
|
S. A.M. Nashef and F. Roques Risk Assessment, League Tables, and Report Cards: Where is True Quality Monitoring? Ann. Thorac. Surg., November 1, 2002; 74(5): 1748 - 1749. [Full Text] [PDF]
|
|
|
|
|
|
R. S.D. Higgins, A. DeAnda, and V. Kasirajan VHI report card gets a failing grade Ann. Thorac. Surg., July 1, 2002; 74(1): 294 - 294. [Full Text] [PDF]
|
|
|
|
|
|
G. L. Grunkemeier, K. J. Zerr, and R. Jin Cardiac surgery report cards: making the grade Ann. Thorac. Surg., December 1, 2001; 72(6): 1845 - 1848. [Full Text] [PDF]
|
|
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
