| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
a Division of Cardiothoracic Surgery, Oregon Health and Science University, Portland, Oregon
b Outcomes Research and Assessment Group, Duke Clinical Research Institute, Durham, North Carolina
c Center for Research in the Implementation of Innovative Strategies in Practice (CRIISP), Iowa City VA Medical Center, Iowa City, Iowa
d Division of General Internal Medicine, Department of Internal Medicine, University of Iowa Carver College of Medicine, Iowa City, Iowa
e Division of Cardiothoracic Surgery, University of Florida, Jacksonville, Florida
f Division of Cardiothoracic and Vascular Surgery, Brody School of Medicine at East Carolina University, Greenville, North Carolina
Accepted for publication June 6, 2007.
* Address correspondence to Dr Welke, Division of Cardiothoracic Surgery, L353, Oregon Health and Science University, 3181 SW Sam Jackson Park Rd, Portland, OR 97239-3098 (Email: welkek{at}ohsu.edu).
 |
Abstract |
|---|
 Top Abstract IntroductionMaterial and MethodsResultsCommentReferences |
|---|
Methods: Hospitals common to both databases were matched. In each database, patients aged 65 years and older who underwent coronary artery bypass grafting, aortic valve replacement, and mitral valve replacement in United States hospitals from 1993 to 2001 were identified.
Results: Annual volumes for all procedures were consistently higher in the NCD. This may be attributed to Medicare managed care; a Medicare group not collected into MedPAR. In-hospital mortality rates trended lower over time and were comparable between the databases. Surgical volumes were generally higher and mortality rates lower for hospitals that submitted data to the NCD than for those that did not.
Conclusions: The close match between NCD and MedPAR in-hospital mortality rates combined with the larger volumes in the NCD suggest that under-reporting in the NCD is not a significant issue. Further investigations into the accuracy of both the NCD and MedPAR are necessary because both are being used for evaluation of provider quality.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
Because of the voluntary nature of the NCD and the lack of a comprehensive audit mechanism, the database has been criticized for being susceptible to provider selection bias with respect to data completeness and accuracy. Although efforts have been made to assure the quality and validity of the NCD [8–10], the lack of a comparable gold standard makes complete corroboration impossible.
The goal of this project is to examine the suitability of the NCD for tracking national cardiac surgery outcomes. There are two specific aims:
|
Material and Methods |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
Each record corresponds to a primary cardiac surgical procedure. Data elements include patient demographics, comorbidities, preoperative risk factors, type of operation, surgical techniques, and outcomes, including in-hospital mortality, 30-day mortality, major morbidity, and postoperative length of stay. Details on data definitions and collection methods, and the semi-annual Executive Summary from the STS NCD Report can be viewed online at http://www.sts.org.
Data from Medicare patients were obtained from the MedPAR Part A public use data files, which were purchased from the Centers for Medicare and Medicaid Services (CMS) and the Health Care Financing Administration. The MedPAR Part A file contains data from claims for services provided to Medicare beneficiaries admitted to Medicare-certified hospitals. Each record represents a beneficiary stay in an inpatient facility and contains data from the Uniform billing-92 (UB-92) hospital discharge abstract. Although the MedPAR file includes 100% of fee-for-service patients, the file may not include patients in Medicare managed care plans, comprising 8%, 15%, and 15% of patients, respectively, in 1995, 1998, and 2001 [11]. Medicare managed care enrollees are younger and healthier than fee-for-service enrollees and have lower hospital use [12, 13]. Thus, the disease burden for patients included in the MedPAR file is likely higher than that of fee-for-service enrollees.
Patients aged 65 years or older who underwent coronary artery bypass grafting (CABG), aortic valve replacement (AVR), mitral valve replacement (MVR), CABG and AVR, or CABG and MVR in US hospitals from 1993 to 2001 were identified in the MedPAR Part A files using International Classification of Diseases, 9th Clinical Modification (ICD-9) codes (AVR: 35.21, 35.22; MVR: 35.23, 35.24; CABG: 36.1x). Patients undergoing any of the following concomitant procedures were excluded from the study population: left ventricular aneurysm repair (ICD-9 37.32), Batista operation (ICD-9 37.35), transmyocardial revascularization (ICD-9 36.31), surgical ventricular restoration (ICD-9 37.35), cardiac transplant (ICD-9 37.5), aortic aneurysm repair (ICD-9 38.45), and carotid endarterectomy (ICD-9 38.12). These operations are more likely to be coded inaccurately than those retained in the analysis. In addition, patients undergoing these operations can differ markedly from the study population and have markedly different mortality rates.
The remaining population was divided into two groups for presentation purposes: (1) the CABG group was CABG without AVR or MVR and (2) the valve group was AVR or MVR with or without CABG. Annual hospital volumes and hospital mortality rates were calculated for each procedure by summarizing the patient level data within each hospital.
To create a comparable sample of patients from the NCD, we identified all patients aged 65 years or older who underwent CABG, AVR, MVR, AVR+CABG or MVR+CABG from January 1, 1993, to December 31, 2001. The same concomitant procedure exclusions were applied to this NCD subset. Although the main provider unit in the NCD is the participant group, for this analysis procedures were analyzed according to the hospital in which they took place.
To identify which hospitals were common to both databases, we matched on hospital name, city, and state. The names and addresses of hospitals in the MedPAR data set were obtained by linking the CMS Provider number in the patient-level MedPAR data set to the name and address of the hospital in the provider-level 2001 CMS Provider file. Among the 1127 unique provider numbers in the patient-level MedPAR data set, 1026 were matched to a hospital name and address in the 2001 CMS Provider file. Nonmatching occurred because the 2001 CMS Provider file only includes provider numbers that were active in 2001. Hospitals that merged or changed ownership between 1993 and 2001 received a new provider number. In such cases, the old number was dropped and only the new number was present in the CMS Provider file.
Among the 1026 MedPAR hospitals with complete hospital name and location data, 626 could be matched to records in the NCD. Nonmatching implied that the hospital did not participate in NCD during the study period or that the hospital participated but did not record its hospital name and location consistently in both databases. Owing to inconsistent spelling and abbreviations of hospital names, some data cleaning and judgment was required to arrive at 626 matched hospitals.
Not all records in the NCD could be matched to a hospital facility in the MedPAR database. Nonmatching occurred when the NCD record had missing or invalid data in the hospital name field (2612 records) or when the NCD record listed a hospital system or surgeon group instead of a hospital (22,630 records). Nonmatching also occurred for 61,177 NCD records that appeared to have valid hospital information (74 hospitals) but had no matching hospital name or location in the MedPAR data set. These 74 unmatched NCD hospitals had a similar geographic distribution to matched hospitals and comparable patient populations, but their case volumes were smaller (median annual CABG volume, 65 versus 144 in 2001; median annual valve volume, 20 versus 34 in 2001). CABG mortality rates of unmatched and matched hospitals were similar (3.90% versus 3.71% in 2001) and valve mortality rates were lower (5.81% versus 7.22% in 2001). Records in the NCD that could not be matched to a hospital in the MedPAR database were deleted at this point and not included in subsequent analyses (73,897 CABG records and 16,057 valve records). Additional matching was done to determine each MedPAR hospital’s participation status in NCD by calendar year.
Hospital institutional characteristics, including organizational structure, personnel, hospital facilities and services, and financial parameters, were determined by matching MedPAR patient records to the 2000 American Hospital Association Annual Survey [14]. The survey data were merged to MedPAR patient records using the Medicare hospital number.
Comparison of National Cardiac Database Participant Hospitals Versus Nonparticipants—Specific Aim 1
We used the 2001 MedPAR database to determine how representative the NCD-participant hospitals were of all US hospitals performing cardiac surgery. To do this we divided hospitals in the MedPAR database into two groups, those that participated (submitted data) in the NCD (NCD-S) and those that did not (NCD-NS). Hospital characteristics, patient characteristics, surgical volumes, and mortality rates for the two populations were then compared.
Comparison of Medicare Provider Analysis and Review Data With National Cardiac Database Data for Participant Hospitals—Specific Aim 2
To determine how accurately the NCD represented the results from participating centers, we compared data for NCD-S hospitals from the NCD with data for the same hospitals from the MedPAR database. We compared patient characteristics for the year 2001, and case volumes and mortality rates (in-hospital and in-hospital or 30-day) for each year between 1993 and 2001. Hospitals that submitted fewer than 6 months of data to the NCD during a calendar year were excluded from the analysis for that year.
We also examined differences between the NCD and MedPAR data sets at the level of individual hospitals to assess whether extreme trends at the hospital level were driving the aggregate analysis. Hospital annual procedural volumes and mortality rates from 2001 data were compared.
|
Results |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
In the MedPAR database, 1,370,000 CABG and 303,689 valve operations were recorded in patients aged 65 and older between 1993 and 2001. The number of procedures varied during the study period, with CABG operations peaking in 1996 and valve operations peaking in 1997. The number of hospitals performing CABG increased from 927 in 1993 to 1029 in 2001 (average of 975 hospitals per year). The number of hospitals in the MedPAR database that submitted data to the NCD increased from 222 (24%) in 1993 to 433 (42%) in 2001 (average of 377 hospitals per year). The percentage of patients undergoing CABG procedures at NCD-S hospitals increased as well, from 23% in 1993 to 44% in 2001. The geographic distribution of NCD-S hospitals was heterogeneous, with the Great Lakes Region having the greatest percentage of institutions submitting data to the NCD (Fig 1).
|
|
|
|
|
|
|
National Cardiac Database Accuracy: Comparison of Medicare Provider Analysis and Review Data With National Cardiac Database Data for Participant Hospitals—Specific Aim 2
Clinical Characteristics
Mean age in the database was similar for CABG (Table 2, columns 2 and 3) and valve operations (Table 3, columns 2 and 3). There was slight variability in gender and race. Comorbidities were generally coded with higher frequency in the NCD database, reflecting both the differences in data definitions and abstraction methods between administrative and clinical data sets and overall data acquisition and quality issues.
Surgical Volume
Surgical volumes were consistently higher in the NCD than in the MedPAR database, particularly in later years (Figs 2 and 3). This trend paralleled the growth of Medicare managed care, which peaked in 2000, and may reflect the omission of patients enrolled in Medicare managed care contract plans from the MedPAR Part A data file. In addition, patients aged older than 65 years who were still employed or were covered by their spouses’ insurance may not have been included in the MedPAR data set if they had not yet enrolled in Medicare.
Hospital level annual CABG volumes were correlated between the two data sets, with a Pearson coefficient of 0.89. The correlation coefficient for valve volumes was 0.90. In 2001, the absolute numerical difference between the volume of CABG cases between the two data sets was less than 10 for 111 hospitals (27.7%) and more than 100 for 40 hospitals (10%). The volume discrepancies were positively correlated with hospital volumes rather than specific hospital characteristics. For 359 hospitals (89.5%), the CABG volume captured in the clinical NCD was greater than that captured in the MedPAR database (data not shown).
Mortality
Annual observed in-hospital mortality rates for isolated CABG were similar between the NCD and the MedPAR database (Table 4). The hospital-level correlation coefficient was 0.79. The absolute difference between CABG mortality rates recorded by the NCD and MedPAR was less than 1.0% for 222 hospitals (55.36%) and larger than 3% for 35 hospitals (8.7%).
|
|
|
Comment |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
When the NCD and MedPAR data for hospitals that participated in the NCD were compared, the MedPAR database reported lower surgical volumes with comparable in-hospital mortality rates. The exclusion of groups of patients, such as those enrolled in Medicare managed care contract plans, from the MedPAR database may account for the volume differences. These patients generally carry less disease burden; therefore, if their omission impacted mortality rates, one would expect to see higher in-hospital mortality rates in the MedPAR database than in the NCD. The fact that in-hospital mortality rates were similar argues against gaming of data by NCD participants. Systematically lower 30-day mortality rates in the NCD reflect the challenges of postdischarge data collection. Although in-hospital mortality rates currently provide the most accurate comparison, future linking of the NCD to the National Death Index or claims data could provide more accurate postdischarge mortality rates.
The increasing discrepancy in case volumes over time suggests that the MedPAR database has become less representative of the national practice of cardiac surgery. In addition, the MedPAR database excludes patients aged younger than 65 years old. According to the NCD, the mean age of patients undergoing CABG in 2004 in the United States was 66; thus, the MedPAR database excludes nearly half of all patients. The NCD may be the best representation of the current practice of cardiac surgery in the United States.
Hospitals participating in the NCD had higher mean surgical volumes and lower mortality than hospitals not participating in the NCD. Hospitals that take part in organized collection, benchmarking, and feedback of clinical data may be more conscious of their practice of care and more likely to engage in other activities directed at quality improvement, despite the associated costs. Although many smaller hospitals participate in the NCD, the resources to support such participation may be more often present at larger hospitals.
The present study supports regional analyses documenting that NCD data are accurate and complete [15, 16]. Comparisons of the NCD with mandatory regional and national clinical databases have demonstrated this as well. Risk factors for surgical mortality and mortality rate trends are similar in the NCD and the national Veterans Affairs database [17], and mortality rates in the Veterans Affairs database, New York State Cardiac Surgery Reporting System database, and the NCD are comparable [18].
This comparison is timely as the CMS and other stakeholders attempt to define the data sources and metrics that will be used to evaluate providers in terms of cost and quality in the pay for performance movement [19]. We chose the MedPAR database as a standard against which to validate the NCD. However, because the databases were created for different purposes and therefore some discrepancy is to be expected, it is difficult to say which served at the gold standard. The UB-92 claim form, from which the MedPAR database is derived, was designed for claims data collection and billing, whereas the NCD was designed for quality improvement [3]. The limited number of diagnostic fields on the UB-92 form may not capture all comorbidities.
In addition, collected data may not be directly relevant to surgical risk adjustment. For example, in the MedPAR database, priority indicates admission priority, not surgical priority as in the NCD. The two terms are not analogous. A patient admitted as an emergency for management of a myocardial infarction may improve and be considered urgent for surgery a week later.
The use of the same diagnostic codes for preoperative comorbidities and surgical adverse outcomes also makes differentiation difficult. Moreover, administrative coding personnel obtain UB-92 data from chart abstraction and considerable variation in the data quality can exist, partly because the coding agenda is often financially driven. Poor documentation in the medical record and limited incentive to clarify data conflicts and fill-in missing data hinder accurate data abstraction.
NCD data are collected by clinical personnel, who have a better knowledge of cardiac surgery than administrative coding personnel. They adhere to standard data definitions. They know where preoperative patient comorbidities are recorded, review operative notes for surgical data, and monitor the patient daily to track complications. The interaction between data personnel across the NCD, the support available from the NCD warehouse team, and the use of collected data for local feedback to improve quality of care provide strong incentives to gather complete and accurate data. As a result, the NCD may be more accurate than the MedPAR database. However, the presumption that physicians will "game the system" through this data collection process has led some to question the validity of the NCD data. Our investigation provides assurance that the NCD is in aggregate accurate and complete, but further inquiry into the accuracy of the collected data is needed to confirm data quality.
This analysis has several limitations. First, the comparison was done at the hospital level. A patient-level analysis, which was not possible because the NCD contains deidentified patient-level data, might reveal additional factors supporting the validity of either database.
Second, the data collection mechanisms for each database precluded matching of some NCD sites with the Medicare database.
Third, owing to the scope of the Medicare population, we could only compare patients aged 65 years or older. Although this comprised 56% of the NCD data set, we found no difference in the completeness of NCD data for patients aged 65 years and older and that for patients younger than 65 (unpublished data).
Finally, the primary limitation to a more rigorous validation of the NCD is the absence of site-level auditing; the STS began a random site-level auditing process in January 2006.
In this comparison with national MedPAR files, we found no evidence of either under-reporting of total case volumes or selective reporting of mortality results in the NCD. Further investigation into the accuracy of both the NCD and MedPAR is necessary, however. In the pay for performance era this is critical, because unvalidated administrative and clinical databases are being used to evaluate providers and provider settings and as information sources for provider quality reports for patients and other stakeholders. The broader age range, more inclusive criteria, and greater clinical detail of the NCD may be more representative of cardiac surgery in the United States than the administrative MedPAR files. In the future, combinations of clinical, financial, and administrative data, including longitudinal data, will give the most accurate representation of the quality, cost, and medical and financial efficacy of cardiac surgical care.
|
References |
|---|
|
TopAbstractIntroductionMaterial and MethodsResultsCommentReferences |
|---|
Related Article
This article has been cited by other articles:
|
|
 |
|
  D. M. Shahian, S. M. O'Brien, G. Filardo, V. A. Ferraris, C. K. Haan, J. B. Rich, S.-L. T. Normand, E. R. DeLong, C. M. Shewan, R. S. Dokholyan, et al. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1--coronary artery bypass grafting surgery. Ann. Thorac. Surg., July 1, 2009; 88(1 Suppl): S2 - 22. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. F. Welke, S. M. O'Brien, E. D. Peterson, R. M. Ungerleider, M. L. Jacobs, and J. P. Jacobs The complex relationship between pediatric cardiac surgical case volumes and mortality rates in a national clinical database. J. Thorac. Cardiovasc. Surg., May 1, 2009; 137(5): 1133 - 1140. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. A. Fleming, K. T. Murray, C. Yu, J. G. Byrne, J. P. Greelish, M. R. Petracek, S. J. Hoff, S. K. Ball, N. J. Brown, and M. Pretorius Milrinone Use Is Associated With Postoperative Atrial Fibrillation After Cardiac Surgery Circulation, October 14, 2008; 118(16): 1619 - 1625. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. L. W. Shroyer Invited commentary Ann. Thorac. Surg., November 1, 2007; 84(5): 1546 - 1547. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
