Ann Thorac Surg 2008;85:1256-1260. doi:10.1016/j.athoracsur.2007.12.013
© 2008 The Society of Thoracic Surgeons
Original Articles: Adult Cardiac
Using the National Death Index to Validate the Noninformative Censoring Assumption of Survival Estimation
YingXing Wu, MD*,
Anthony P. Furnary, MD,
Gary L. Grunkemeier, PhD
Medical Data Research Center, Providence Health System, Portland, Oregon
Accepted for publication December 4, 2007.
* Address correspondence to Dr Wu, 9205 SW Barnes Rd, Ste 33, Portland, OR 97225 (Email: yingxing.wu{at}providence.org).
 |
Abstract
|
|---|
Background: In survival analysis, a patient who is missing complete follow-up is included in the analysis as a censored observation. The analysis makes the assumption that the censoring is noninformative; that is, that a censored patient has the same risk of death as those who have complete follow-up. We tested this assumption in a large, long-term follow-up study.
Methods: From 1986 through 2003, 14,495 patients underwent isolated coronary artery bypass grafting procedures. Of 13,963 eligible patients, 2312 were lost to follow-up. We obtained National Death Index data to complete our follow-up, and then compared survival between the original data and the complete National Death Index–augmented data.
Results: The National Death Index data revealed 855 additional deaths and increased the total follow-up years from 86,810 to 102,157. Survival estimates and regression models did not differ between the original and National Death Index–augmented data.
Conclusions: Patients lost to follow-up might not differ with regard to survival from those with complete data. The requirement for 95% completeness is somewhat arbitrary. The quality and type of follow-up is more important than the percentage in time-related analyses.
|
Introduction
|
|---|
When a survival curve is produced, the censoring of lost patients is assumed to be "noninformative"; that is, their subsequent survival will be the same as those who are not censored. However in general, it is not possible to construct a statistical test for noninformative censoring [1, 2], and these lost patients may produce biased estimates if their prognosis does differ from the noncensored patients [3]. It is for this reason that the percentage of completeness should be as high as possible, and the completeness of follow-up is often used as a measure of quality of follow-up (sometimes this is the sole measure) obtained in a published study. Its importance has been stressed; for example, the American Association for Thoracic Surgery/Society of Thoracic Surgeons guidelines for reporting recommend at least 95% completeness for heart valve studies [4–6].
We have maintained a prospective lifetime follow-up service for open heart surgery patients since 1960 but have found it difficult to achieve higher than an 85% to 90% completeness of follow-up. Are lost patients indeed a different subset? And if so, does follow-up really need to be 95% complete to make accurate statistical inference?
|
Patients and Methods
|
|---|
From 1986 through 2003, 14,495 patients underwent isolated (no concomitant valve procedure) coronary artery bypass grafting (CABG) procedures at Providence St. Vincent (PSV) Medical Center. Of these, 306 patients (2.1%) died before hospital discharge, and the 14,189 patients discharged alive were entered into our follow-up service and prospectively contacted at annual intervals. Patients are flagged as "lost" in our cardiac database if they are last known to be alive but did not respond to a mailed questionnaire and attempted telephone interview during the previous 3 years. By this definition, 2229 patients were lost as of the end of 2003. We obtained National Death Index (NDI) information on 2312 patients: 2003 of the lost patients (226 lost patients were inadvertently not submitted to NDI) plus 309 patients who were not lost (to compare our information to that of the NDI).
The NDI returns information in three categories (Appendix). If a true match was determined by NDI, then the patient’s status was set to "dead," and the latest date was set to the NDI death date; if there was a false match or no match, the vital status was set to "alive," and the last day of 2003 was used as the latest date. Because there is a 2-year lag for validated NDI data to appear, the latest NDI data available at the time of this study (2005 to 2006) was December 31, 2003. To make our data comparable with the NDI data, our patient follow-up data were also truncated at that date.
The study excluded the 226 patients who were lost but were not submitted to NDI, and the remaining 13,963 operative survivors were included in the analysis. To determine whether the lost patients had different survival than patients who were not lost, that is, whether their censoring was informative, we compared the survival curves of the 11,651 patients whose complete follow-up and deaths were obtained by our PSV system and the 2312 lost patients whose latest follow-up and deaths were obtained from NDI.
Different methods have been used to calculate the percentage of completeness [7, 8], and for the same data set, the results can be quite different. Wu and colleagues [9] proposed a method (C*) that calculated the completeness as a ratio of observed follow-up time and expected survival time.
The NDI data gave us an opportunity to validate the C* statistic [9] and also to test the noninformative censoring assumption by comparing the long-term results using our incomplete patient follow-up with the results based on NDI-completed data. To find out if increasing the completeness of follow-up would affect statistical inference of long-term survival, we compared the long-term survival of all 13,963 patients using the original (PSV only) follow-up with that using the NDI-augmented (PSV + NDI) follow-up. We also compared the univariate and multivariable effects of risk factors on long-term survival using the original and the augmented follow-up. Survival curves were produced by the Kaplan-Meier method [10]. Cox regression models [11] were used to derive hazard ratios for covariate effects on long-term survival, and Schoenfeld residuals were used to test proportional hazards for each covariate [12].
|
Results
|
|---|
As of the end of 2003, the 13,963 CABG patients experienced 3627 documented deaths and accumulated 86,810 follow-up years (mean, 6.2; maximum, 18.0 years) in the PSV follow-up system. The percentage completeness using the C* statistic [9] was 84.5%. The NDI identified 855 additional deaths (true matches), 712 false matches, and 742 no matches (Appendix). For the 309 patients who were not lost but were submitted to the NDI, all but 2 matched the vital status in our follow-up system. The NDI added 15,347 follow-up years. This brought the true follow-up completeness to C* = 100% for the 2312 patients submitted to NDI, and to C* = 98.4% for the entire series of 13,963 patients (not 100%, because our "not-lost" patients, not submitted to NDI, did not all have complete follow-up through the end of 2003).
Survival of Complete (Providence St. Vincent Only) vs Lost (National Death Index Only) Patients
Table 1
compares some risk factors for long-term survival between the 11,651 complete patients and the 2312 lost patients. The complete patients were older, fewer were women, and had more hypertension and more unstable angina; the lost patients had more renal dysfunction, more history of myocardial infarction, more diseased vessels, and advanced New York Heart Association functional class. Yet the two groups had comparable long-term survival: their Kaplan-Meier survival curves almost overlap, both overall (Fig 1) and for subsets defined by quartiles of surgery year (Fig 2).
View larger version (16K):
[in this window]
[in a new window]
|
Fig 1. Survival for the 11,651 patients in our system who were complete (not lost; 98% complete, gray line), and the 2312 patients who were lost (black line) and whose death status was obtained from National Death Index (100% complete).
|
|
View larger version (27K):
[in this window]
[in a new window]
|
Fig 2. Survival for complete (gray line) and (lost, black line) patients compared by year of surgery quartiles.
|
|
Survival of Original (Providence St. Vincent) vs. Augmented (Providence St. Vincent and National Death Index)
Combining the complete and lost groups yields an NDI-augmented group of 13,963 patients. We compared this with the original data (PSV only) for the same group of patients. Long-term survival was virtually identical for these two data sets, both overall (Fig 3) and for subsets defined by quartiles of surgery year (Fig 4). Moreover, the quantitative effects of the risk factors for mortality were virtually unchanged for these two data sets (original PSV only vs NDI augmented), both by univariate (Fig 5) and multivariable (Fig 6) Cox regressions. The hazard ratios and their 95% confidence intervals are almost identical between the two data sets. The proportional hazards assumption held for all the risk factors in this model.
View larger version (19K):
[in this window]
[in a new window]
|
Fig 3. Survival using the original Provident St. Vincent (PSV) only data (complete plus lost, gray line), compared with the National Death Index (NDI)–augmented data (PSV plus NDI, black line). The original data were 84.5% complete, and the augmented data were 100% complete.
|
|
View larger version (29K):
[in this window]
[in a new window]
|
Fig 4. Survival using the original Provident St. Vincent (PSV) data (gray line) and the National Death Index (NDI)–augmented data (black line) by year of surgery quartiles.
|
|
View larger version (8K):
[in this window]
[in a new window]
|
Fig 5. Hazard ratios of the risk factors in the univariate Cox regression models developed by using the original Provident St. Vincent data (open circles) and the National Death Index (NDI)-augmented data (solid circles). The horizontal lines are the 95% confidence intervals of the hazard ratios. The letters correspond to the risk factors listed in Table 1. (A = age; B = female; C = prior coronary artery bypass grafting; D = diabetes; E = history of renal dysfunction; F = chronic obstructive pulmonary disease; G = hypertension; H = history of myocardial infarction; I = unstable angina; J = New York Heart Association class III or IV; K = number of diseased vessels.)
|
|
View larger version (7K):
[in this window]
[in a new window]
|
Fig 6. The hazard ratios of the risk factors in the multivariable Cox regression models developed by using the original Provident St. Vincent data (open circles) and the National Death Index (NDI)–augmented data (solid circles). The horizontal lines are the 95% confidence intervals of the hazard ratios. The letters correspond to the risk factors listed in Table 1. See Fig 5 for definitions of the letters.
|
|
|
Comment
|
|---|
The NDI data allowed us to test the assumption that the censoring of lost patients is noninformative and evaluate the potential gains of using NDI data as a supplement to regular ongoing prospective follow-up. The NDI information increased the completeness of follow-up to virtually 100% but did not affect the statistical inference of the survival data at all. The Kaplan-Meier survival curves and Cox regression models for survival are almost identical for the original incomplete data and the NDI-augmented data. It is usually assumed that patients lost to follow-up are not a random sample from the study patients, and the attempt to contact those patients failed because they were sicker and more likely to be dead. The opposite could also be true, however. The nonresponding patients could be healthier than average, and thus denying their disease, ignoring their former caregivers, or actively retired and on an extended vacation.
The lost patients and complete patients may have different clinical profiles, but they have a similar risk of mortality. The truth is, at least in this relatively large, long-term experience of isolated CABG patients who were actively followed up at annual intervals, that the survival prognosis of lost patients matches almost exactly that of their counterparts with complete data, and the censoring of their lifetimes is truly noninformative. Whether this is also applicable to data sets with less frequent follow-up schedules or different types of follow-up (for example outpatient visits) requires further exploration.
The study tested the noninformative assumption for long-term survival after CABG. It may not hold for other end points, such as stroke. The assumption should also be tested on patients who have valve operations, or even outside the cardiac surgery population, in future studies.
The NDI provides an efficient and economic method of identifying the vital status of study subjects. Several studies have shown that the results found by NDI are quite accurate; however, the rates varied [13–16]. Some studies rely exclusively on the NDI to provide their outcomes because it is much less labor-intensive and much less costly at $0.21 per subject per year of death search from NDI, but approximately $4.83 per patient per year of follow-up by our system. If NDI can give the answer, why should investigators spend money and keep dedicated personnel to run their own prospective, ongoing follow-up system?
The NDI provides only death information, which is very important to most clinical studies, but cannot provide information about nonfatal complications, which is also of great interest in many studies. For nonfatal events, the completeness of the data will not be increased by the NDI. From this study, we cannot deduct whether the cumulative incidence estimates would change or not if the follow-up were completed for nonfatal events. In the assessment of nonfatal events, one would expect that quality of follow-up (frequency and type) rather than completeness of follow-up is even more crucial compared with fatal events, because the human mind is designed to quickly forget unfortunate happenings.
In conclusion, the study validates the assumption of noninformative censoring required for the use of actuarial methods and suggests that patients lost to follow-up might not differ with regard to survival from those with complete data. Sources of bias can be due to either different methods of follow-up being used for different comparative groups or to the likelihood of treatment effects being associated with the ease of tracing patients. Whether this is also true for nonfatal events still needs evaluation. We conclude that the requirement for 95% completeness is somewhat arbitrary, and that the quality of follow-up is more important than the percentage of follow-up completeness. Reporting guidelines should emphasize providing details on the quality of follow-up and recommend methods to standardize the calculation of completeness.
|
Appendix
|
|---|
The National Death Index
The concept of a National Death Index (NDI) was proposed by a Subcommittee of the National Committee on Vital and Health Statistics in 1968 [17], and a computer index for death has been available since 1979. Instructions on applying for NDI search, preparing data, and interpreting NDI reports can be found on the National Center for Health Statistics Web site (http://www.cdc.gov/nchs/ndi.htm).
The NDI matches user records to death records based on 12 criteria such as Social Security Number, name, sex, and date of birth. Weights for each of the 12 items used for assessing the quality of potential match were constructed based on the composition of the 1988 to 1991 National Health Interview Survey (NHIS) and 1986 to 1991 United States deaths. A score classifying potential matches is computed as the sum of the weights.
The potential matches are classified into one of five mutually exclusive classes based on which items match and the number of items matching. All of class 1 matches are considered to be true matches implying that the individual is deceased. All of class 5 matches are considered false matches implying that the individuals are not yet deceased. Class 2, 3, and 4 matches are either true matches or false matches. Records with scores greater than the cutoff scores are considered true matches; records with scores lower than the cutoff scores are considered false matches.
|
Acknowledgments
|
|---|
We are indebted to Vicki Hargrove for her dedication to obtaining prospective PSV follow-up data on our cardiac surgery patients and to Vicki Anderson for obtaining and processing the additional NDI follow-up data.
|
References
|
|---|
- Lagakos SW. General right censoring and its impact on the analysis of survival data Biometrics 1979;35:139-156.[Medline]
- Leung KM, Elashoff RM, Afifi AA. Censoring issues in survival analysis Annu Rev Public Health 1997;18:83-104.[Medline]
- Sica GT. Bias in research studies Radiology 2006;238:780-789.[Abstract/Free Full Text]
- Edmunds Jr LH, Clark RE, Cohn LH, Grunkemeier GL, Miller DC, Weisel RD. Guidelines for reporting morbidity and mortality after cardiac valvular operations. Ad Hoc Liaison Committee for Standardizing Definitions of Prosthetic Heart Valve Morbidity of The American Association for Thoracic Surgery and The Society of Thoracic Surgeons. J Thorac Cardiovasc Surg 1996;112:708-711.[Free Full Text]
- Edmunds Jr LH, Clark RE, Cohn LH, Grunkemeier GL, Miller DC, Weisel RD. Guidelines for reporting morbidity and mortality after cardiac valvular operations. The American Association for Thoracic Surgery, Ad Hoc Liaison Committee for Standardizing Definitions of Prosthetic Heart Valve Morbidity. Ann Thorac Surg 1996;62:932-935.[Abstract/Free Full Text]
- Edmunds Jr LH, Clark RE, Cohn LH, Grunkemeier GL, Miller DC, Weisel RD. Guidelines for reporting morbidity and mortality after cardiac valvular operations Eur J Cardiothorac Surg 1996;10:812-816.[Abstract]
- Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials Lancet 2001;357:1191-1194.[Medline]
- Clark TG, Altman DG, De Stavola BL. Quantification of the completeness of follow-up Lancet 2002;359:1309-1310.[Medline]
- Wu Y, Takkenberg JJM, Grunkemeier GL. Measuring follow-up completeness Ann Thorac Surg 2008;85:1155-1157.[Abstract/Free Full Text]
- Kaplan EL, Meier P. Nonparametric estimation from incomplete observations J Am Stat Assoc 1958;53:457-481.
- Cox DR. Regression models and life-tables (with discussion) J R Stat Soc 1972;B34:187-220.
- Schoenfeld DA. Partial residuals for the proportional hazards regression model Biometrika 1982;69:239-241.[Abstract/Free Full Text]
- Curb JD, Ford CE, Pressel S, Palmer M, Babcock C, Hawkins CM. Ascertainment of vital status through the National Death Index and the Social Security Administration Am J Epidemiol 1985;121:754-766.[Abstract/Free Full Text]
- Calle EE, Terrell DD. Utility of the National Death Index for ascertainment of mortality among cancer prevention study II participants Am J Epidemiol 1993;137:235-241.[Abstract/Free Full Text]
- Lash TL, Silliman RA. A comparison of the National Death Index and Social Security Administration databases to ascertain vital status Epidemiology 2001;12:259-261.[Medline]
- Edlavitch SA, Baxter J. Comparability of mortality follow-up before and after the National Death Index Am J Epidemiol 1988;127:1164-1178.[Abstract/Free Full Text]
- National Committee on Vital and Health Statistics Use of vital and health records in epidemiologic research. Public Heath Service Pub. No 1000, Series 4, No 7. Washington DC: US Govt Printing Office; 1968.
Related Article
-
Measuring Follow-Up Completeness
- YingXing Wu, Johanna J.M. Takkenberg, and Gary L. Grunkemeier
Ann. Thorac. Surg. 2008 85: 1155-1157.
[Abstract]
[Full Text]
[PDF]
This article has been cited by other articles:
|
|
|
|
 
Y. Wu, J. J.M. Takkenberg, and G. L. Grunkemeier
Measuring Follow-Up Completeness
Ann. Thorac. Surg.,
April 1, 2008;
85(4):
1155 - 1157.
[Abstract]
[Full Text]
[PDF]
|
|
|
Top
Abstract
Introduction
