HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Barry, S. J.E. Articles by McKhann, G. M. Search for Related Content PubMed PubMed Citation Articles by Barry, S. J.E. Articles by McKhann, G. M. Related Collections Coronary disease

Ann Thorac Surg 2005;79:1104-1109
© 2005 The Society of Thoracic Surgeons

---

The statistician's page

Quantitative Methods for Tracking Cognitive Change 3 Years After Coronary Artery Bypass Surgery

^a Department of Biostatistics, The Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
^b Department of Neurology, The Johns Hopkins University School of Medicine, Baltimore, Maryland
^c Department of Surgery, The Johns Hopkins University School of Medicine, Baltimore, Maryland
^e Department of Neuroscience, The Johns Hopkins University School of Medicine, Baltimore, Maryland
^d Zanvyl Krieger Mind Brain Institute, The Johns Hopkins University, Baltimore, Maryland

^_* Address reprint requests to Ms Barry, Department of Biostatistics, The Johns Hopkins School of Public Health, 615 N Wolfe St, Baltimore, MD 21205 (E-mail: sbarry{at}jhsph.edu).

	Abstract

Top Abstract Introduction Analyze, Then Summarize Results Comment Acknowledgments References

 
The analysis and interpretation of change in cognitive function test scores after coronary artery bypass grafting (CABG) present considerable statistical challenges. Application of hierarchical linear statistical models can estimate the effects of a surgical intervention on the time course of multiple biomarkers.

We use an "analyze then summarize" approach whereby we estimate the intervention effects separately for each cognitive test and then pool them, taking appropriate account of their statistical correlations. The model accounts for dropouts at follow-up, the chance of which may be related to past cognitive score, by implicitly imputing the missing data from individuals' past scores and group patterns. We apply this approach to a study of the effects of CABG on the time course of cognitive function as measured by 16 separate neuropsychological test scores, clustered into 8 cognitive domains. The study includes measurements on 140 CABG patients and 92 nonsurgical controls at baseline, and at 3, 12, and 36 months.

Our "analyze then summarize" method allows us to identify differences between the treatment groups in individual tests as well as in aggregate measures. It takes into account the correlation structure of the data and thereby produces more precise results than summarizing before analyzing.

The methods used have application to a wide range of intervention studies in which multiple biomarkers are followed over time to quantify health effects. Software to implement the methods in the R statistical package is available from the authors at http://www.biostat.jhsph.edu/sbarry/software/ATSrcode.pdf.

	Introduction

Top Abstract Introduction Analyze, Then Summarize Results Comment Acknowledgments References

 
Current approaches to the analysis of cognitive test data in establishing an association between coronary artery bypass graft (CABG) surgery and cognitive change generally use a "summarize then analyze" methodology in which results of potentially disparate tests are summarized before analysis takes place [1, 2]. This approach potentially masks important results from individual tests, allowing only study of aggregate measures. It ignores the correlation structure of the cognitive test data, leading to imprecision in the results.

We present an "analyze then summarize" approach in which tests are analyzed individually before summarization of the results of these initial analyses, taking into account correlation within people over time and across tests. This method allows us to study both the individual and aggregate results.

As an illustration of our methodology, we will apply it to data from an ongoing study that allows comparison of patients receiving CABG with a group of patients who have established coronary artery disease, but do not have surgery; these nonsurgical controls (NSC) have an incidence of risk factors for vascular disease similar to that of the CABG group. In the accompanying paper by Selnes and colleagues [3], we compare the longitudinal performance of these two groups at baseline and at 3, 12, and 36 months after surgery or enrollment.

A first question in this example is whether the pattern of cognitive change in the CABG group differs from that observed in the NSC group. A second question is whether any differences are likely caused by the surgery.

In this paper, we discuss a hierarchical statistical model [4] that can be used to quantify differences in change in cognitive function over time between the CABG and control groups. We use the statistical model to estimate the average cognitive function performance on each test over time for the surgery and control groups, after adjusting for known differences in potential confounding variables, specifically age, sex, education, and the presence of symptoms of depression.

Finally, we use bootstrapping [5] to combine the estimates of the surgery effects across many measures into domain-specific estimates of group differences. This method uses prior knowledge about the domains of cognitive function measured by each test as well as the correlation structure of the data.

The statistical approaches we have used for evaluation of prospective, longitudinal data comparing patients after CABG with a nonsurgical control group have general applicability to other clinical studies in which there are multiple outcomes and the goal is the evaluation of the impact of an intervention.

	Analyze, Then Summarize

Top Abstract Introduction Analyze, Then Summarize Results Comment Acknowledgments References

 
We present in this section an approach that analyzes the responses from each individual test separately using hierarchical models. The test responses are initially standardized and adjusted for demographics before fitting the models. The estimated regression coefficients from the separate models are then summarized using a weighted average determined by the correlation structure of the individual tests. We use bootstrapping to obtain standard errors of the combined regression coefficients. The following sections explain the stages in detail.

Analyze
STUDY DESIGN
The data used to illustrate our method come from an observational study of 140 patients undergoing CABG and 92 nonsurgical cardiac controls. Surgical patients (CABG) and nonsurgical controls were recruited from September 1997 through March 1999 at the Johns Hopkins Cardiac Unit. The NSC group was identified by Johns Hopkins cardiologists as potential patients who were diagnosed with coronary artery disease by cardiac catheterization.

Study participants were administered a battery of standardized neuropsychological tests at baseline and at 3, 12, and 36 months. Patients were also administered the Center for Epidemiological Studies Depression scale (CES-D) at baseline and follow-up [6], in order to adjust cognitive test scores for possible effects of depressed mood. (See the accompanying paper by Selnes and coworkers [3] for a detailed description of the patient population and study design.)

HIERARCHICAL LINEAR STATISTICAL MODEL
This section describes a now standard statistical model [4] designed to capture the key components of the change in cognitive function over time for the individuals in our study and for their population, and to compare the typical change for persons who do and do not receive an intervention such as CABG. As an example, we focus on a single cognitive domain, verbal memory, that constitutes four tests; below we present a method for summarizing results across tests to obtain the domain values.

The model is specified by the following assumptions: (1) Each person has a unique level and time trend of cognitive function. (2) Over periods of time, such as a few years, true cognitive function changes gradually and can be approximated by a smooth function of time, such as a low order polynomial [7]. (3) The intervention may affect people in the short term by immediately increasing or decreasing their function; and over the longer term by changing their preintervention trend. The short-term and long-term effects of intervention may vary across individuals. (4) The level of cognitive function is influenced, possibly in a nonlinear way, by other factors such as age, sex, education, and level of depressive symptoms.(5) Measurements of cognitive function are subject to a practice effect whereby a study participant's scores on quantitative tests could improve with repetition, particularly from the first to second testing, absent a change in actual cognitive function level.

Figure 1 presents a schematic of this model. The goal is to estimate the effects of an intervention from a dataset comprising repeated observations on cognitive tests over time for persons who have received the intervention and other similar persons who have not.

View larger version (14K): [in this window] [in a new window]   Fig 1. Schematic representation of the statistical model for estimating the effects of an intervention (eg, coronary artery bypass graft surgery) on a single measure of cognitive function. The oval represents an unobserved variable. See the Appendix for details. (CESD = Center for Epidemiological Studies Depression scale.)

The proposed model has two degrees of freedom to quantify a possible effect of CABG: the rise from 0 to 3 months (short-term, or learning effect); and a difference in the slope from 3 to 36 months (long-term, or time trend effect). We use a Wald test of the null hypothesis of no CABG effect, which simultaneously tests whether the two regression coefficients are equal to zero by assuming that the coefficients are approximately normally distributed in large samples and comparing the relevant test statistic to the ² distribution [8].

When dropout is related to past cognitive score, individuals seen at all follow-up points may have a different distribution of scores to the entire group, and this is likely to cause bias. The model has the ability to reduce this bias by using information from previous timepoints and group patterns to internally impute missing data at later follow-up points and thereby make more precise estimates of the true group means at follow-up of all the individual patients who started the study.

ESTIMATING NATURAL HETEROGENEITY
In addition to estimating the mean curves for each intervention group, the model is used to estimate the amount of variation in the true levels and time trends of a cognitive test score among persons within groups [7]. We anticipate some variation in the baseline scores and change over time between people purely by reason of individual persons being heterogeneous; therefore, we expect scores from a particular patient over time to be more similar than those between people. The model takes into account and estimates this correlation among repeated observations on an individual person. We allow the degree of variation to be estimated separately for the two intervention groups to capture any extra variation that may arise in the treatment group as a result of the intervention differentially benefiting or harming subjects.

Summarize
The hierarchical model in the Appendix is estimated separately for each of the 16 measures of cognitive function. This produces, for each measure, short- and long-term intervention effect estimates from the learning effect and time trend group differences and a 2 x 2 covariance matrix that quantifies their statistical error. We estimate the mean learning or time trend "effect" for a domain as the weighted mean of the effects for the tests in that domain, where the weights depend on the correlation structure. Since the multiple test scores for an individual are correlated with one another, estimates of the CABG effects for the different measures are also correlated. To correctly estimate the standard errors of these domain-specific or overall effects, we must take this correlation into account. We use bootstrapping [5], a statistical method that involves resampling individuals, to estimate the joint covariance matrix among the 16 pairs of intervention effect estimates and to obtain valid standard errors for the domain and overall intervention effect estimates. We draw with replacement a random sample of 140 CABG and 92 NSC subjects, refit all 16 models to get test-specific learning and time trend effect estimates, average these to obtain domain and overall effect estimates, and then repeat this process 1,000 times. The variance among the 1,000 bootstrapped replicates of the domain and overall effect estimates gives a valid estimate of statistical uncertainty, used to calculate the confidence intervals of the effect estimates, as it takes appropriate account of the correlation among multiple cognitive test scores for the same individual person.

	Results

Top Abstract Introduction Analyze, Then Summarize Results Comment Acknowledgments References

 
We now apply our method to the example data. The analyses were performed using the R software package [9].

Figure 2 is a "spaghetti" plot of the standardized and covariate-adjusted scores over time for the Verbal Memory domain stratified by intervention group. The cognitive test scores were standardized such that the NSC group had a mean score of zero and standard deviation of 1 at baseline, and were adjusted for age, sex, education level, and depressive symptoms. The Verbal Memory domain is made up of the total score, delayed recall (Trial 8), retention score, and corrected recognition from the Rey Auditory Verbal Learning Test [12]. The group mean scores at each time are also shown. For graphical simplicity the plot is drawn using the 3-, 12-, and 36-month time points rather than the actual times at which patients were tested. The model may be fitted using the actual times if desired, but since our testing times were close to 3, 12, and 36 months, the assumption of these exact times is minor. A learning effect is evidenced by the increase in mean score in both groups from baseline to 3 months. There is little change from 3 to 36 months in the mean response for either group after the initial rise.

View larger version (46K): [in this window] [in a new window]   Fig 2. Spaghetti plots of the individual standardized and adjusted (for age, gender, education, and depressive symptoms) Verbal Memory scores across time (grey lines), stratified by treatment group with the solid black lines showing the group means. The dashed lines represent constant zero score as a reference. Nonsurgical control group (left). Coronary artery bypass graft surgery group (right).

View larger version (13K): [in this window] [in a new window]   Fig 3. Illustration of the steps taken to fit the hierarchical model to the Verbal Memory data. (A) Average curves for each intervention group; solid lines are coronary artery bypass graft surgery group, and dashed lines are the nonsurgical control group. (B) Average change from baseline for each intervention group. (C) Difference in mean change from baseline between the coronary artery bypass graft surgery group and the nonsurgical control group. (D) Model estimates of group difference in change from baseline that take appropriate account of missing data.

Panel C is a plot of the difference between the intervention (CABG) and control group (NSC) curves in panel B at each time. These differences are the essential evidence relevant to assessing the intervention (CABG) effect, as they show the disparities between the cognitive scores of the group that has had surgery and a group with similar risk factors that has not. Without this comparison, we cannot attribute any change in scores for the CABG group to the surgery. They still require some adjustment, however, because they are averages of observed data and do not take account of the fact that the number of dropouts may differ between the two groups.

Panel D presents improved estimates of the difference at each time between the CABG and NSC group curves shown in panel C. The results in panel D are obtained from the hierarchical model in the Appendix. The small differences in the curves in panels C and D reflect changes from taking appropriate account of missing data. In the case of this domain, the CABG patients remaining at 36 months are those with poorer cognitive scores at baseline than the patients who dropped out, with the opposite effect in the NSC group, a phenomenon likely to bias the mean group score at 36 months toward lower values. The hierarchical model implicitly imputes the missing data by using the information for the individual patients at previous times and the patterns for their group. The means in panel C ignore the missing data and are biased unless the chance of dropping out is unrelated to past cognitive score, which is an unlikely situation.

Panel D shows evidence of a difference in population mean Verbal Memory value between the intervention and control groups, with the CABG group having a greater improvement from baseline than the NSC group, both in the short term at 3 months and in the long term at 36 months. The test of the null hypothesis that both the short- and long-term effects are zero has a p value of 0.01, indicating that this hypothesis can be rejected in favor of the CABG group.

Figure 4 shows the estimated difference in the cognitive function time course between the intervention and control groups (panel D in Fig 3 for Verbal Memory) for all 8 of the cognitive domain measures. The accompanying paper by Selnes and associates [3] presents the corresponding figure for the 16 cognitive subtests. The p values on the plots in Figure 4 again result from tests of zero difference in the trends over time between the groups and make it clear that in this dataset, there is little or no evidence consistent with a detrimental effect of CABG on cognitive function as measured by these 8 scores.

View larger version (27K): [in this window] [in a new window]   Fig 4. Model estimates of the difference in cognitive function change between the coronary artery bypass graft surgery group and nonsurgical control group over time for all eight domains, where a positive difference in the solid line indicates a greater improvement from baseline in the surgery group than in the nonsurgical control group. The dashed lines show constant zero difference in score and represent the difference in change from baseline between the nonsurgical control group and itself (hence zero). Note that where there is only one test in a domain, the estimate for that particular test is used instead of the bootstrap estimate.

	Comment

Top Abstract Introduction Analyze, Then Summarize Results Comment Acknowledgments References

The model described here estimates the average difference between the CABG and control groups in the change in cognitive function from baseline. We use the model to adjust for baseline differences between the groups in test scores and for differences over time that are attributable to demographics and depression symptoms, the latter as measured by the CESD, as well as natural heterogeneity between people.

However, no model can adjust for unmeasured differences between the two groups that are more likely to arise in observational studies where subjects choose their treatment in consultation with their physician rather than having it assigned by a known, random mechanism. Hence, we must be cautious in our interpretation of the evidence, asking what other factors might account for the differences or lack thereof between the two groups.

The hierarchical model allows one to take appropriate account of dropouts, a common phenomenon in longitudinal studies such as this one. The model includes terms that acknowledge the correlation among repeated observations for each individual. Having done so, it can internally impute missing values by predictions based upon the earlier responses and other covariates [7]. Failure to use a model that accounts for within-person correlation can lead to biased estimates of treatment effects except when the dropout process is independent of the past responses, which is unlikely. A limitation of the method, however, is the assumption that patients would have followed their initial trend since most of the missing data are at the 36-month follow-up point.

We have presented an approach to the difficult problem of how to estimate the effect of CABG on the performance of 16 cognitive measures by first analyzing each of them separately with an hierarchical model and then pooling the effect estimates to obtain domain effects. We refer to this as the "analyze then summarize" approach. An alternative is to "summarize then analyze" the data by using factor analysis [10] or some other method to create summary scores from the 16 test results and then to use hierarchical models with the summary measures. We prefer our approach because it produces a separate treatment effect for every measure so that unanticipated patterns can be discovered. An example of the serious limitations of the "summarize then analyze" approach could occur when combining the results of two tests; one of which shows a decline over time in the treatment group while the other shows an improvement, but for both tests the control group remains stable. Summarizing before analyzing would allow the treatment effects to be canceled out, and we would have no way of identifying them.

Another approach to this problem is repeated measures analysis of variance (ANOVA). For this problem, we prefer our method because repeated measures ANOVA includes only a random intercept and therefore has a less flexible model of the covariance structure among the repeated observations [7]. A further, commonly used approach to the problem involves the use of "SD" methods, whereby patients are classified as having cognitive decline if they, for example, are 1 standard deviation worse at follow-up in 20% of the tests than they were at baseline [11]. We believe, however, that purely by chance some patients will do worse at follow-up than at baseline and likewise some patients will improve. To consider only those who are declining at follow-up misses important information about patients who are improving and who on average may balance out the amount of change the groups are showing. This method also fails to quantify the type of correlation between repeated observations that can be of interest in its own right.

Furthermore our method avoids the difficult problem of how to choose the best summary scores. Typically, summarization is based upon the correlation among test results at one time and does not take appropriate account of the longitudinal information.

The methods used here have wide application to a variety of longitudinal studies comparing intervention groups or groups defined in other ways when there are multiple outcomes. To facilitate the application of these methods, software to implement the analyses presented here has been posted to our Webpage (available at: http://www.biostat.jhsph.edu/sbarry/software/ATSrcode.pdf).

	Appendix

 
Hierarchical Statistical Model for a Single Biomarker
This Appendix specifies the hierarchical linear statistical model to quantify the difference in average time-curve for intervention and control groups and to estimate an individual person's time curve, acknowledging the natural heterogeneity among persons in the level and trend of the cognitive functioning. To be precise, we make the following statements:

A. Cognitive function = Intercept + covariate effects at baseline + test-retest improvement for second test and beyond + linear time trend.

B. Intercepts differ between treatment groups; test-retest improvements differ between treatment groups; linear trends differ between treatment groups.

C. Subjects vary witin treatment groups in their true: intercepts; time trends.

D. The degree of subject-to-subject variation in the intercepts and slopes (in C.) differs between the two treatment groups.

R program code specifying this model is available on the web at: http://www.biostat.jhsph.edu/sbarry/software/ATSrcode.pdf.

	Acknowledgments

Top Abstract Introduction Analyze, Then Summarize Results Comment Acknowledgments References

 
This study was supported by Grant 35610 from the National Institute of Neurologic Disorders and Stroke, National Institutes of Health, Bethesda, Maryland; the Charles A. Dana Foundation, New York, New York; and The Johns Hopkins Medical Institution GCRC Grant RR 00052, Baltimore, Maryland. Special thanks are extended to Maryanne Bailey, Catherine Cristinzio, Sarah Moeller, and Sharon Owens, who assisted in the neuropsychological assessments, and to our study participants who volunteered their time and energy to make this study possible. We thank Pamela Talalay, PhD, for her help during the preparation of this manuscript. We would also like to thank the cardiologists (Drs Nicholas Fortuin, Steven P. Schulman, Roy Ziegelstein, and David Meyerson), cardiac surgeons (Drs R. Scott Stuart, William A. Baumgartner, Marc S. Sussman, Levi Watkins, Duke E. Cameron, Peter S. Greene, J. Mark Redmond, and John V. Conte), and the anesthesiologists at our institution.

	References

Top Abstract Introduction Analyze, Then Summarize Results Comment Acknowledgments References

 

1. Newman MF, Kirchner JL, Phillips-Bute B, et al. Longitudinal assessment of neurocognitive function after coronary-artery bypass surgery N Engl J Med 2001;344:395-402.[Abstract/Free Full Text]
2. Stygall J, Newman SP, Fitzgerald G, et al. Cognitive change 5 years after coronary artery bypass surgery Health Psychol 2003;22:579-586.[Medline]
3. Selnes OA, Grega MA, Borowicz LM Jr, et al. Cognitive outcomes three years after coronary artery bypass surgery: a comparison of on-pump coronary artery bypass graft surgery and nonsurgical controls. Ann Thorac Surg 2005;79:1201–9..
4. Laird NM, Ware JH. Random-effects models for longitudinal data Biometrics 1982;38:963-974.[Medline]
5. Efron B, Tibshirani R. Improvements on cross-validation: the Bootstrap method J Am Stat Assoc 1997;92:548-560.
6. Radloff LS. The CES-D scale: a self-report depression scale for research in the general population Appl Psychol Measurement 1977;1:385-401.
7. Diggle PJ, Heagerty P, Liang KY, Zeger SL. The analysis of longitudinal data2nd ed. Oxford, England: Oxford University Press; 2002.
8. Agresti A. Categorical data analysis. 2nd ed. Hoboken, NJ: Wiley-Interscience, 2002..
9. R Development Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2004. Available at: http://www.R-project.org. Accessed November 2004..
10. DeVellis RF. Scale development: theory and applicationsNewbury Park, CA: SAGE Publications; 1991.
11. Mahanna EP, Blumenthal JA, White WD, et al. Defining neuropsychological dysfunction after coronary artery bypass grafting Ann Thorac Surg 1996;61:1342-1347.[Abstract/Free Full Text]
12. Rey A. L'examen clinique en psychologie. Paris: Presses Universitaires de France, 1964. .

This article has been cited by other articles:

				O. A. Selnes, M. A. Grega, M. M. Bailey, L. Pham, S. Zeger, W. A. Baumgartner, and G. M. McKhann Neurocognitive Outcomes 3 Years After Coronary Artery Bypass Graft Surgery: A Controlled Study Ann. Thorac. Surg., December 1, 2007; 84(6): 1885 - 1896. [Abstract] [Full Text] [PDF]

				H. J. Nathan, R. Rodriguez, D. Wozny, J.-Y. Dupuis, F. D. Rubens, G. L. Bryson, and G. Wells Neuroprotective effect of mild hypothermia in patients undergoing coronary artery surgery with cardiopulmonary bypass: Five-year follow-up of a randomized trial J. Thorac. Cardiovasc. Surg., May 1, 2007; 133(5): 1206 - 1211. [Abstract] [Full Text] [PDF]

				J. R. Keith, D. J. Cohen, and L. B. Lecci Why Serial Assessments of Cardiac Surgery Patients' Neurobehavioral Performances are Misleading Ann. Thorac. Surg., February 1, 2007; 83(2): 370 - 373. [Full Text] [PDF]

				I. Szalma, A. Kiss, L. Kardos, G. Horvath, E. Nyitrai, Z. Tordai, and L. Csiba Piracetam prevents cognitive decline in coronary artery bypass: a randomized trial versus placebo. Ann. Thorac. Surg., October 1, 2006; 82(4): 1430 - 1435. [Abstract] [Full Text] [PDF]

				O. A. Selnes, L. Pham, S. Zeger, and G. M. McKhann Defining Cognitive Change After CABG: Decline Versus Normal Variability Ann. Thorac. Surg., August 1, 2006; 82(2): 388 - 390. [Full Text] [PDF]

				G. M. McKhann, M. A. Grega, L. M. Borowicz Jr, M. M. Bailey, S.J.E. Barry, S. L. Zeger, W. A. Baumgartner, and O. A. Selnes Is there cognitive decline 1 year after CABG?: Comparison with surgical and nonsurgical controls Neurology, October 11, 2005; 65(7): 991 - 999. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Barry, S. J.E. Articles by McKhann, G. M. Search for Related Content PubMed PubMed Citation Articles by Barry, S. J.E. Articles by McKhann, G. M. Related Collections Coronary disease