HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): YingXing Wu Johanna J.M. Takkenberg Gary L. Grunkemeier Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Wu, Y. Articles by Grunkemeier, G. L. Search for Related Content PubMed PubMed Citation Articles by Wu, Y. Articles by Grunkemeier, G. L. Related Collections Education Related Article

---

The Statistician's Page

Measuring Follow-Up Completeness

^a Medical Data Research Center, Providence Health System, Portland, Oregon
^b Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, the Netherlands

^_* Address correspondence to Dr Wu, 9205 SW Barnes Rd, Suite 33, Portland, OR 97225 (Email: yingxing.wu{at}providence.org).

	Abstract

Top Abstract Introduction Patients and Methods Results Comment References

 
Completeness of follow-up is often used as a measure of the quality of follow-up, but the method used to compute it is often not declared. An ideal measure should be based on follow-up years instead of patients. Clark, Altman, and De Stavola proposed such a measure, called "C", which is the percentage of the maximum possible follow-up years, as of a given date, that has actually been accounted for or observed. However, such a measure will underestimate the true completeness, because the denominator (maximum possible follow-up years) does not account for unobserved patient deaths occurring before that date, and therefore, it is realistically unachievable. We propose a modification, C*, of Clark’s C, which accounts for the effect of unobserved patient deaths in attenuating the maximum potential follow-up, and thus gives a higher percentage for achieved follow-up completeness. We validated this theoretical improvement by comparing the values of C and C* computed for our long-term coronary artery bypass graft patients to the true completeness, which was obtained by using the National Death Index to complete our missing follow-up data. Using Clark’s C, the follow-up completeness was 80.4% and using our C* it is 84.5%, whereas the true follow-up completeness based on National Death Index information was 85.0%.

	Introduction

Top Abstract Introduction Patients and Methods Results Comment References

 
In long-term medical and surgical cohort studies, patients are followed up over time to document whether and when they have certain events of interest, such as death. There are always patients who are lost to follow-up. Statistically, the lifetimes of these lost patients are treated as incomplete or censored values. However, lost patients may produce biased estimates. Therefore, the completeness of follow-up is often used as a measure of the quality of follow-up. Many articles report completeness of follow-up; however, the definition of percentage of completeness is often not provided.

One popular method is to consider as complete all patients who had follow-up during a certain terminal period (eg, the last year or 2 before the closing date of the study), plus all of the known deaths. The percentage of follow-up completeness is then computed by dividing this number of complete patients by the total number of patients in the study. However, definitions based only on numbers of patients do not take the duration of follow-up time into consideration. A patient who has been in the study for 20 years and who has been followed up for 18 years would be lost, whereas a patient who has been enrolled for 3 years and followed up for only 2 years would be complete.

Clark and colleagues [1] proposed a measure of completeness, called C, which does take follow-up years into account. Clark’s C is the ratio of the total observed follow-up years divided by the total potential follow-up years. The observed follow-up for each patient is the time from study entry (T₀) to the date of last contact (T₁). The potential follow-up for each patient is the time from study entry (T₀) to the closing date of the study (T₂) or to patient death, whichever comes first. Clark’s method is an improvement compared with a simple percentage of patients, but the problem with this definition is that the denominator of the ratio, or the potential follow-up time, is overestimated, because some lost patients would have died before the closing date (T₂), especially for longer series, older patients, and diseases with high mortality. Thus, because the denominator is overestimated, Clark’s C ratio is an underestimate of the true fraction of follow-up that could realistically be achieved.

We propose a modification of Clark’s method, called C*, which does take these anticipated deaths into account. For each patient who is last known to be alive, instead of adding the entire time from T₁ to T₂, we add the expected survival time from T₁ to T₂ to obtain potential follow-up time (see Appendix). Data completed by the National Death Index (NDI), applied to our coronary artery bypass graft patients show that C* provides a better estimate of the true completeness than C.

	Patients and Methods

Top Abstract Introduction Patients and Methods Results Comment References

 
From 1986 through 2003, 14,495 patients underwent isolated (no concomitant valve procedure) coronary artery bypass graft procedures at Providence St. Vincent Medical Center. There were 14,189 patients discharged alive who were entered into our lifetime follow-up service and prospectively contacted at annual intervals by mailed questionnaire or telephone interview, or both. Patients were flagged as lost in our cardiac database if they were last known to be alive, but did not respond to our follow-up attempt during the previous 3 years. By this definition, 2,229 patients were lost as of the end of 2003. We obtained NDI information on 2,003 of these lost patients, and 226 lost patients were inadvertently not submitted to the NDI and were excluded from this study, leaving 13,963 patients in the study. Detailed information on using NDI data is explained in a companion publication [2].

	Results

Top Abstract Introduction Patients and Methods Results Comment References

 
As of the end of 2003, the 13,963 coronary artery bypass graft patients had experienced 3,627 documented deaths and accumulated 86,810 follow-up years (mean, 6.2 years; maximum, 18.0 years) through December 2003 in the Providence St. Vincent Medical Center follow-up system. The values of C, C*, and the true (NDI) follow-up completeness are given, using computations and abbreviations from the Appendix.

Clark’s C
For our coronary artery bypass graft patient data, the observed follow-up for all 13,963 was:

where OBS is the observed follow-up, and the maximum potential missing follow-up for all patients from their date of last contact until December 31, 2003 was:

where ADD = T₂–T₁ and the observed survival time is T₁ and the potential follow-up time is T₂, so that the completeness statistic of Clark and colleagues [1], given by equation A1 in the Appendix, is:

The Proposed Modification, C*
There were 3,480 observed deaths among all patients for a death rate (r) of 0.040/year. Using this, the adjusted potential follow-up was:

where 5,199 is the follow-up years inevitably lost by the deaths that occur between T1 and T2, and the adjusted follow-up completeness statistic, given by equation A2 in the Appendix, is:

True (NDI) Completeness
The NDI identified 855 additional deaths (true matches) and added 15,347 follow-up years. This brought the true follow-up completeness to 100% for the 2,312 patients submitted to the NDI, and to 98.4% for the entire series of 13,963 patients (not 100%, because our "not-lost" patients, not submitted to the NDI, which did not all have 100% complete follow-up through the end of 2003). Thus, the true follow-up completeness using the NDI augmented data was:

	Comment

Top Abstract Introduction Patients and Methods Results Comment References

 
The CONSORT guidelines [3] recommend a simple count of participants lost to follow-up. Clark’s C method, based on percentage of follow-up years, is an improvement compared with the CONSORT method. However, Clark’s method is based on the assumption that all the patients who were not known to be dead can live to the closing date of the study, and thus underestimates the percentage of completeness. Siskind [4] pointed out the weakness of Clark’s method, but he did not suggest a better method. We propose a new measure, C*, which is a modification of C.

To estimate the potential survival time, we assumed a constant death rate. However, the risk of death increases with age. The Gompertz model has been shown to provide an excellent fit, but would require extensive computation. We compared the potential survival times predicted by the Gompertz and exponential (constant rate) distributions, and they differed by only a small amount (0.4%) overall.

The NDI was established to assist medical and health investigators to obtain vital status of study subjects, especially for studies requiring long-term follow-up. The NDI augmented data gave us an opportunity to complete our missing follow-up data and test how well the new measure C* estimated the true percentage of follow-up completeness. The follow-up is 84.5% complete by the new measure C*, and this was only slightly (0.5%) different from the true value of 85.0% complete using the NDI augmented data. Clark’s C is 80.4%, which indeed underestimated the true completeness.

	Appendix

 
Quantifying the Percentage of Follow-Up Completeness

We present the calculations for the method C of Clark and colleagues [1] and our proposed modification of C*. Equation numbers in brackets are cited within the text.

Calculations
Clark and colleagues’ [1] defines the measure of completeness as:

where OBS is the observed follow-up and MAX is the maximum potential follow-up if all living patients were alive on the closing date of the study. This denominator could be expressed as

is the added follow-up for the living patients, from their date of last contact (T₁) to the closing date of the study (T₂). That is,

(Eq. A1)

This definition assumes that none of the living patients die before T₂. But, if the true follow-up were known for all patients at the closing date, not all of them would still be alive, so the true percentage of completeness would be larger than C. An estimate of the follow-up years lost to these deaths can be obtained by assuming, as an approximation, a constant death rate. If the death rate (r) is constant, then the proportion surviving at "t" years after surgery is:

where exp() is the exponential function. For a patient who has already survived to time T₁, the conditional future survival curve for t > T₁ is given by S(t)/S(T₁). The expected lifetime experienced by this patient during the time from T₁ to T₂ is the area under this conditional survival curve, which (by integration) is equal to:

The first 3 terms of the Taylor series expansion of an exponential function is:

Using this approximation,

Note that ADD* equals ADD minus a positive quantity. This quantity could be considered to represent a subtraction for the follow-up years inevitably lost by the deaths that occur before T₂. Thus, the sum of ADD* is the years of follow-up to be added to OBS, to correct for the anticipated deaths before T₂. That is,

(Eq. A2)

C* will be larger than C, because the denominator is smaller due to the subtraction of the "maximum potential" patient years that are not actually observed, which are due to the expected and inevitable patient deaths of some lost patients before the closing date of the study.

	References

Top Abstract Introduction Patients and Methods Results Comment References

 

1. Clark TG, Altman DG, De Stavola BL. Quantification of the completeness of follow-up Lancet 2002;359:1309-1310.[Medline]
2. Wu Y, Furnary AP, Grunkemeier GL. Using the National Death Index to validate the noninformative censoring assumption of survival estimation Ann Thorac Surg 2008;85:1256-1260.[Abstract/Free Full Text]
3. Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials Lancet 2001;357:1191-1194.[Medline]
4. Siskind V. Quantification of completeness of follow-up Lancet 2002;360:724.[Medline]

This article has been cited by other articles:

				K. Ng, S. Ogino, J. A. Meyerhardt, J. A. Chan, A. T. Chan, D. Niedzwiecki, D. Hollis, L. B. Saltz, R. J. Mayer, A. B. Benson III, et al. Relationship Between Statin Use and Colon Cancer Recurrence and Survival: Results From CALGB 89803 J Natl Cancer Inst, October 19, 2011; 103(20): 1540 - 1551. [Abstract] [Full Text] [PDF]

				D. Niedzwiecki, M. M. Bertagnolli, R. S. Warren, C. C. Compton, N. E. Kemeny, A. B. Benson III, S. G. Eckhardt, S. Alberts, G. N. Porjosh, D. J. Kerr, et al. Documenting the Natural History of Patients With Resected Stage II Adenocarcinoma of the Colon After Random Assignment to Adjuvant Treatment With Edrecolomab or Observation: Results From CALGB 9581 J. Clin. Oncol., August 10, 2011; 29(23): 3146 - 3152. [Abstract] [Full Text] [PDF]

				K. Ng, J. A. Meyerhardt, J. A. Chan, D. Niedzwiecki, D. R. Hollis, L. B. Saltz, R. J. Mayer, A. B. Benson III, P. L. Schaefer, R. Whittom, et al. Multivitamin Use Is Not Associated With Cancer Recurrence or Survival in Patients With Stage III Colon Cancer: Findings From CALGB 89803 J. Clin. Oncol., October 1, 2010; 28(28): 4354 - 4363. [Abstract] [Full Text] [PDF]

				B. I. Rini, S. Halabi, J. E. Rosenberg, W. M. Stadler, D. A. Vaena, L. Archer, J. N. Atkins, J. Picus, P. Czaykowski, J. Dutcher, et al. Phase III Trial of Bevacizumab Plus Interferon Alfa Versus Interferon Alfa Monotherapy in Patients With Metastatic Renal Cell Carcinoma: Final Results of CALGB 90206 J. Clin. Oncol., May 1, 2010; 28(13): 2137 - 2143. [Abstract] [Full Text] [PDF]

				C. Lee, C.-H. Lee, J. G. Kwak, C. S. Park, S.-J. Kim, J. Y. Song, and W.-S. Shim Long-term results after mitral valve repair in children Eur J Cardiothorac Surg, February 1, 2010; 37(2): 267 - 272. [Abstract] [Full Text] [PDF]

				W. R. E. Jamieson, R. Koerfer, C. A. Yankah, A. Zittermann, R. I. Hayden, H. Ling, R. Hetzer, and W. B. Dolman Mitroflow aortic pericardial bioprosthesis -- clinical performance, Eur J Cardiothorac Surg, November 1, 2009; 36(5): 818 - 824. [Abstract] [Full Text] [PDF]

				Y. Wu, A. P. Furnary, and G. L. Grunkemeier Using the National Death Index to Validate the Noninformative Censoring Assumption of Survival Estimation Ann. Thorac. Surg., April 1, 2008; 85(4): 1256 - 1260. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): YingXing Wu Johanna J.M. Takkenberg Gary L. Grunkemeier Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Wu, Y. Articles by Grunkemeier, G. L. Search for Related Content PubMed PubMed Citation Articles by Wu, Y. Articles by Grunkemeier, G. L. Related Collections Education Related Article