| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Ann Thorac Surg 1997;63:1531-1532
© 1997 The Society of Thoracic Surgeons
Division of Cardiothoracic Surgery, University of Colorado Health Sciences Center, Denver, Colorado
Estimating risk for operative death, morbidity, and other outcomes is becoming increasingly important to cardiothoracic surgeons and others in the medical field. This allows us to come closer to measuring quality of care by taking into account patient risk factors. The article by Drs Lippmann and Shahian [1] is an excellent review of several statistical methods for predicting risk-adjusted outcomes. Although it is somewhat complex because of the statistical language, it is very important because it furthers our knowledge of statistical methodology in this evolving area. Of particular interest is the fact that the more complex neural network analysis does not at the present time offer any greater predictive accuracy than does the Bayesian analysis or logistic regression analysis.
Lippmann and Shahian examined 80,606 patients from The Society of Thoracic Surgeons (STS) Database who underwent operation during 1993 and compared logistic regression and the Bayesian analysis with single-, two-, and three-layer neural networks. These were evaluated by equal members of training and cross-validation testing groups generating receiver operating characteristic curves for predicting mortality. The receiver operating characteristic curve areas, or C-indices, were approximately 76% for all of the techniques, but there was some improvement in accuracy as measured by the C-index when this was combined with the best neural network and the logistic regression model, although this difference was not a large one. Of interest is the fact that when the patients were stratified into six risk levels (0% to 2.5%, 2.5% to 5%, 5% to 10%, 10% to 20%, 20% to 30%, and 30% to 100%) each of the methods examined was quite accurate for the lowest three sextiles but the Bayesian system considerably overestimated the risk for the high-risk patients, the logistic regression somewhat overestimated the risk, and the two-layer neural network method underestimated the risk. Because the two-layer neural network method underestimated the high-risk patients but was relatively accurate otherwise and the logistic regression was also relatively accurate but overestimated the high-risk group, these two groups were combined and yielded the best accuracy.
Although the statistics in this article's methodology and results are somewhat complex for those of us who are cardiothoracic surgeons, the Comment section of the article is superb and offers a very nice review of the strengths and limitations of risk assessment statistical analysis, which all of us should have a knowledge of.
Doctor Guillermo Marshal (formerly with our group in Denver) and associates [2] performed a similar analysis on eight statistical methods and found that cluster analysis and stepwise logistic regression analysis had the highest C-indices of 0.71, with Bayesian at 0.695 and logistic regression at 0.694. They later combined the Bayesian methodology and logistic regression technique and found that the Bayesian model and the logistic regression model had C-indices of 0.695, 0.701 and 0.710, respectively, all being very close in their predictive accuracy. The combination of the Bayesian and the logistic regression offered some slightly increased accuracy [3].
The receiver operating characteristic curves or C-indices are used very commonly to evaluate the accuracy of these models. A value of 1.0 means total predictability; 0.5 is equivalent to total chance, or a flip of a coin. Most of these methods have a predictability about half way between chance and total predictability [4].
The Bayesian model was initially adopted by the STS Database Committee because of its reliability, flexibility, and forgiveness of databases that have a considerable number of missing values [5]. Over the course of the almost 10-year history of the STS database, however, missing values have diminished significantly. We have now begun using the logistic regression analysis because missing values are less frequent and in general the logistic regression methodology performs slightly superior to other methods. It is therefore reassuring to those of us on the STS Database Committee that Drs Lippmann and Shahian's neural network technique does not offer any significant improvement over the logistic regression. It is, however, interesting that their combination of the logistic regression with one of the neural network techniques offers greater predictability in the high-risk group, and we therefore read this article with interest.
We have recently completed the annual 1996 harvest of the STS Database. There are now 894,223 patients entered, and the annual reports were distributed at the February meeting of the STS [6]. For the first time the Canadian data have been risk stratified and there will be a separate United States and Canadian report as well as a combined report. Because the risk estimates are only as good as the data entered into the database, there are a number of processes that are being implemented to measure and improve the quality of data entry. The STS has hired Mrs Mary Eiken, RN, MS, as a national database coordinator who will work with the local data managers on issues such as definitions, completeness, and quality of data. We conducted our first database manager symposium at the annual meeting in San Diego, offering background educational material as well as hands-on chart abstraction in an attempt to decrease variability between various database managers in their interpretation of definitions and abstraction of cases. We have also reviewed the variations in the reporting of risk factors and mortality and morbidity by the more than 500 surgical groups and have asked for an explanation from those groups that significantly vary from the norm. Site visits are anticipated to several of these locations to clarify these variations and hopefully eliminate them.
Risk assessment will be available this coming year for valve operations and combined valve and coronary operations in addition to coronary artery bypass grafting. We are also initiating a collection of data specific for minimally invasive cardiac operations. We are attempting to develop a more realistic, user-friendly format for the congenital database and are hoping to increase participation in the general thoracic database so that we can risk stratify those patients.
A very important activity of the Database Committee has been getting information up on the STS Web page, and we are now promoting dialogue regarding our definitions on the Web page. It may be that eventually we will have direct computer entry for the database, and we can then update the national averages on the Web page for our membership.
Another effort toward improving the credibility of the STS Database is the obtaining of expert statistical consultation by the appointment of an expert advisory panel, which will meet this spring.
We are also working with the American College of Cardiology in standardizing our definitions and are hopeful that the Health Care Financing Administration and several states, which have approached us with interest regarding our database, will use this database for the measurement of quality and efficiency both regionally and nationally.
It is extremely important at this critical point in the changing health care environment that we have accurate, validated, and respected data that reflect the quality of our work and its cost effectiveness. Many of the major decisions that are being made in the health care area at this time are being made largely on the basis of cost rather than quality. It is our ethical obligation to closely monitor our surgical outcomes to be sure that these cost-cutting policies are not adversely affecting patient care [7].
For this reason we welcome and applaud Drs Lippmann and Shahian's work as a step forward in the area of outcomes research as we continually strive to improve the accuracy of our database and to make it more useful to our membership.
Footnotes
Address reprint requests to Dr Grover, Division of Cardiothoracic Surgery (C-310), University of Colorado Health Sciences Center, 4200 E Ninth Ave, Denver, CO 80262.
References
Related Articles
This article has been cited by other articles:
|
|
 |
|
  D. M. Shahian, S.-L. Normand, D. F. Torchiana, S. M. Lewis, J. O. Pastore, R. E. Kuntz, and P. I. Dreyer Cardiac surgery report cards: comprehensive review and statistical critique Ann. Thorac. Surg., December 1, 2001; 72(6): 2155 - 2168. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
