ATS
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


This Article
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Personal Folders
Right arrow Download to citation manager
Right arrow Author home page(s):
Richard J. Novick
Bob B. Kiaii
W. Douglas Boyd
Right arrow Permission Requests
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Novick, R. J.
Right arrow Articles by Boyd, W. D.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Novick, R. J.
Right arrow Articles by Boyd, W. D.
Related Collections
Right arrow Coronary disease

Ann Thorac Surg 2002;73:S358-S362
© 2002 The Society of Thoracic Surgeons


SUPPLEMENT: OUTCOMES 2001

Assessing the learning curve in off-pump coronary artery surgery via CUSUM failure analysis

Richard J. Novick, MD*a,b, Stephanie A. Fox, RRCPa, Larry W. Stitt, MSb, Bob B. Kiaii, MDa,b, Stuart A. Swinamer, MDa, Reiza Rayman, MDa, Thomas R. Wenske, MDa, W. Douglas Boyd, MDa,b

a London Health Sciences Center, London, Ontario, Canada
b University of Western Ontario, London, Ontario, Canada

* Address reprint requests to Dr Novick, Division of Cardiac Surgery, Room D228, London Health Sciences Center, South Street Campus, 370 South St, London, Ontario, Canada N6B1B8
e-mail: rjnovick{at}uwo.ca

Presented at Outcomes 2001, "The Key West Meeting," Key West, FL, May 23–27, 2001.

Despite mounting evidence of the benefits of off-pump coronary artery bypass grafting (OPCAB) [1], many surgeons are reluctant to switch from on-pump coronary artery bypass to OPCAB. One of the reasons that there may be concern that a shift from coronary artery bypass grafting on cardiopulmonary bypass (CPB) to OPCAB may be associated with deleterious patient outcomes, especially during the learning phase. We have previously shown that use of the sequential probability cumulative sum (CUSUM) failure technique to analyze surgical outcomes is more sensitive than standard statistical methods in detecting a cluster of failures or successes [2]. Recently, we applied CUSUM methods to evaluate the learning curve associated with a policy change by a single surgeon from on-pump coronary artery bypass grafting to multivessel OPCAB [3]. The results of this CUSUM analysis showed clear evidence of benefit and a reduction in adverse patient outcomes in the initial weeks and months after the policy change.

The purpose of the current study was to compare patient outcomes associated with the first author’s practice of coronary artery bypass grafting on CPB versus OPCAB from July 1999 to March 2001. First, we wished to determine whether patients selected for OPCAB exhibited different preoperative characteristics than those who underwent an on-pump operation. Second, by analyzing patient outcomes using standard statistical tests as well as the CUSUM method, we wished to determine the ability of the latter technique to more sensitively reflect surgical outcomes in a comparison of the two different patient cohorts. Third, we decided to evaluate the relationship between the type of coronary artery bypass procedure and postoperative length of hospital stay using univariable and multivariable analyses, to determine if OPCAB was an independent predictor of a shorter length of hospital stay postoperatively.

Patients and methods

Inclusion and exclusion criteria
Before March 2000, the author performed OPCAB only when the patient required a single left internal thoracic artery bypass to the left anterior descending coronary artery. During that month, the author decided to approach every patient undergoing first-time, multivessel coronary artery bypass grafting as a potential candidate for OPCAB. Subsequently, all patients were considered candidates for OPCAB unless they had a critical left main coronary artery stenosis in concert with disease in the right coronary artery, required five or more bypass grafts, or had deeply buried or diffusely calcified target vessels. The goal in OPCAB patients was to achieve complete revascularization, albeit with a low threshold for conversion to CPB to maximize patient safety during the learning curve.

Criteria for inclusion in this study were consecutive patients undergoing first-time elective or urgent coronary artery bypass grafting from July 1999 until March 2001. Exclusion criteria included patients undergoing redo coronary artery bypass procedures, patients in cardiogenic shock preoperatively, or patients requiring an emergency operation shortly after surgical referral.

Operative technique and definitions of mortality and morbidity
The operative technique employed by the author for on-pump coronary artery bypass grafting and OPCAB has been recently reviewed elsewhere [3]. The Cor-Vasc System (Coroneo, Montreal, Quebec, Canada) was used for coronary artery stabilization in all OPCAB cases. The operational definitions of mortality and morbidity in this study were identical to those reported in the author’s two recent papers, which focused on cardiac surgical learning curves [2, 3]. In this study, we analyzed the risk of mortality and 10 major complications in the two cohorts of patients undergoing either coronary artery bypass grafting on CPB or OPCAB. These major complications included perioperative myocardial infarction, use of the intraaortic balloon pump, reoperation due to bleeding, new renal failure, stroke, sternal dehiscence, mediastinitis, sepsis, respiratory insufficiency, and postoperative cardiac arrest/arrhythmia. The last complication, which was not reported in our previous studies, denoted the requirement for cardioversion at any time postoperatively for an atrial or ventricular arrhythmia, or the necessity for permanent postoperative cardiac pacing.

Statistical methods
Preoperative patient risk factors and postoperative results were compared between the CPB and OPCAB groups using Fisher’s exact tests for categorical variables and two-tailed t-tests or Wilcoxon two-sample tests for continuous variables (depending on whether between group variances were equal). Analysis of variance was used to compare operative variables in four quartiles of sequential OPCAB patients. Multiple comparisons were made using Tukey’s test, where applicable. Linear regression was used to determine factors associated with the duration of hospital stay postoperatively. Step-wise linear regression allowing for entry and removal at the 0.20 level was used to determine factors independently associated with postoperative hospital stay. A p value of less than 0.05 was considered statistically significant.

Use of the CUSUM method
In addition to these tests, the CUSUM method was used to chart the changes in patient outcomes associated with coronary artery bypass grafting on CPB and OPCAB. The theoretical advantages of the CUSUM technique over standard statistical methods have been reviewed elsewhere [2, 3]. In short, the CUSUM technique employs a sequential probability ratio test, thus recognizing the importance of time as a "hidden variable" in clinical studies [4] and avoiding the statistical problems associated with repeated significance testing [5]. Moreover, the CUSUM method detects clusters of surgical failures during the patient care process rather than after an arbitrary unit of time, thus alerting the surgeon to suboptimal results in advance of standard statistical measures. In this study, CUSUM was defined as Sn = {Sigma}(Xi–Xo), where Xi = 1 for a "failure" (death or any of the ten major complications) and Xi = 0 for a "success" (avoidance of death or any major complication), whereas Xo is the reference value [2, 3, 6]. Xo was set at 0.1, indicating that the "maximum acceptable failure rate" (ie, combined rate of death or any of the 10 major complications) was 10% [2, 3]. CUSUM curves were constructed manually to document the pattern of cumulative failures during the performance of coronary artery bypass grafting on CPB versus OPCAB from July 1999 to March 2001. Furthermore, 80% alert and 95% alarm boundary lines were calculated and drawn according to formulae presented in the appendices of the authors’ recent papers [2, 3].

Results

During the 21 months of the study, 248 patients underwent first-time elective or urgent coronary artery bypass grafting by the first author. In this patient cohort, 160 patients underwent coronary artery bypass grafting on CPB (CPB group), whereas 88 underwent OPCAB (OPCAB group).

The preoperative characteristics of the patients in the CPB and OPCAB groups are shown in Table 1. These groups were well matched and exhibited no statistically significant differences in any of the risk variables. Patients in the CPB group received a mean of 3.3 ± 0.6 grafts per patient versus 2.9 ± 0.6 grafts per patient in the OPCAB group (p < 0.001).


View this table:
[in this window]
[in a new window]
 
Table 1. Preoperative Characteristics of Patients in the CPB (n = 160) and OPCAB (n = 88) Groups

 
Table 2 shows the number of grafts and operating times per patient in four sequential quartiles of OPCAB cases. The number of grafts per patient increased significantly after the first quartile of experience. The operating time per patient significantly increased between quartiles one and two, due to the increased number of grafts per patient in the latter quartile.


View this table:
[in this window]
[in a new window]
 
Table 2. Operative Variables in OPCAB Cases by Quartile (n = 22 Patients Each)

 
Table 3 shows the incidence of adverse patient outcomes in the CPB and OPCAB groups. No patient in the OPCAB group experienced postoperative mortality or stroke. Standard statistical analyses showed no significant differences between groups in mortality or any of the 10 listed complications, although the difference in the incidence of respiratory failure between groups was close to achieving statistical significance. Seven of 88 patients (8%) in the OPCAB group experienced one or more of the adverse outcomes noted in Table 3 versus 19 of 160 patients (11.9%) in the CPB group (p = 0.39).


View this table:
[in this window]
[in a new window]
 
Table 3. Postoperative Outcomes of Patients in the CPB (n = 160) and OPCAB (n = 88) Groups

 
As shown in Table 4, measures of hospital length of stay tended to be shorter in OPCAB, as opposed to CPB patients, although the differences did not achieve statistical significance. To evaluate better the possible contribution of OPCAB to a reduction in postoperative length of hospital stay, univariable and multivariable linear regression models were constructed for this outcome variable. The results of these models are shown in Table 5. OPCAB was close to being significantly associated with a reduced postoperative hospital stay on univariable analysis, but was not independently predictive of a decreased length of stay in the multivariable model. Increasing age, a preoperative diagnosis of peripheral vascular disease, and a previous transient ischemic attack or stroke were independently predictive of an increased postoperative length of stay in this series.


View this table:
[in this window]
[in a new window]
 
Table 4. Duration of ICU and Hospital Stay in CPB (n = 160) and OPCAB (n = 88) Patients

 

View this table:
[in this window]
[in a new window]
 
Table 5. Factors Associated With and Independently Predictive of Postoperative Length of Hospital Stay in the 248 Coronary Bypass Patients

 
The CUSUM curves of patients undergoing coronary artery bypass grafting on CPB and OPCAB are shown in Figure 1. The author’s OPCAB experience was associated with a low risk of cumulative failures from the outset, and the curve in OPCAB patients gravitated around or below the lower 80% alert line (ie, reassurance boundary). On the other hand, the CUSUM curve in CPB patients transgressed the upper 80% alert line after 25 and 32 cases, and then gravitated to the mid zone between the lower and upper 80% alert lines, before reaching below the lower 80% (reassurance) boundary after 85 cases. These highly sensitive CUSUM data demonstrated differences in outcome between the CPB and OPCAB groups, despite the fact that the standard statistical analyses presented in Tables 3 and 4 showed that these differences were not yet statistically significant.



View larger version (18K):
[in this window]
[in a new window]
 
Fig 1. CUSUM analysis of clinical experience with on-pump and off-pump (OPCAB) coronary bypass surgery. X-axis denotes consecutive patients undergoing either procedure from July 1999 to March 2001. Y-axis denotes the number of adjusted cumulative failures (death or any of 10 prespecified major complications), assuming a total "acceptable failure rate" of 10%. Formulae for construction of 80% alert lines and 95% alarm lines are shown in the appendices of the author’s two recent papers [2, 3]. A CUSUM curve that reaches below the lower 80% alert line (ie, reassurance boundary) indicates results better than target values.

 
Comment

The results of this study showed that the adoption of OPCAB was not associated with adverse clinical sequelae, and indeed showed some evidence of benefit on CUSUM analysis, as compared with coronary artery bypass grafting on CPB. None of the OPCAB patients died or had a stroke, and only 8% experienced any of the 10 complications listed in Table 3. On the other hand, patients undergoing coronary artery bypass grafting on CPB in this study had satisfactory clinical results that were comparable with what has been reported in the Society of Thoracic Surgeons database [7] and in single institution analyses [8].

Interestingly, OPCAB surgery was not an independent predictor of a shorter hospital length of stay on multivariable analysis in our study, even though data shown in Table 4 indicated that OPCAB patients spent on average 2 days less in the hospital postoperatively than CPB patients. The fact that other variables aside from OPCAB were independent predictors of postoperative length of stay implies that these variables were of greater importance in predicting postoperative outcomes than the type of coronary bypass procedure. The three independent predictors of postoperative length of hospital stay in our study were, in order of significance, the presence of peripheral vascular disease, increasing age, and a history of transient ischemic attack or stroke (Table 5). As noted in Table 1, the ages and neurologic histories of the patients in the CPB and OPCAB groups were almost identical. Twice as many patients in the CPB group had a diagnosis of peripheral vascular disease preoperatively than in the OPCAB group, although this difference did not achieve a p value less than 0.10. These data indicate that there is the possibility of a type II error in comparing an important preoperative variable such as peripheral vascular disease between the two groups, in view of the fact that the study cohort contained only 248 patients. A larger study may have shown statistically significant differences in some preoperative characteristics between the CPB and OPCAB groups. Moreover, the importance of peripheral vascular disease as a new independent predictor of mortality after coronary artery bypass grafting has been highlighted in a recent publication from another Canadian center [9].

Although OPCAB now counts for more than 20% of coronary artery bypass operations in the United States [1], there have been few randomized controlled trials comparing OPCAB with on-pump coronary bypass surgery. The Bristol Heart Institute Group demonstrated less renal dysfunction, a less intense systemic inflammatory response, less troponin release, fewer arrhythmias, less postoperative infection, and a lower cost in patients randomized to OPCAB, as opposed to coronary artery bypass grafting on CPB [1013]. Published observational studies have presented less uniform results, although the majority have reported less blood loss, fewer blood transfusions, and less inotrope and balloon pump use in OPCAB, as compared with CPB patients [1]. Moreover, observational data from our own institution have shown a reduction in neuropsychological dysfunction at 3 months postoperatively from 50% in CPB patients to 5% in OPCAB patients [14]. A recently published study from the Minneapolis group did not demonstrate a major difference in outcomes between OPCAB and CPB patients, except in the highest risk group of patients, who realized a significant mortality benefit from OPCAB [15]. These data illustrate the importance of further randomized controlled trials comparing on-pump coronary artery bypass grafting with OPCAB, to better guide patient selection for these procedures.

This study confirms the usefulness of CUSUM analysis as a complementary technique to standard statistical outcome measures. By sensitively detecting changes in patient outcomes during the patient care process, CUSUM curves can alert the surgeon to suboptimal performance in advance of standard statistical analyses. Moreover, a priori construction of 80% alert and 95% alarm lines enables the surgeon to determine whether the outcomes of innovative procedures are acceptable or need to be improved. In higher risk procedures than coronary artery bypass grafting, the "maximum acceptable failure rate" in the CUSUM formula can be increased in order to better reflect clinical realities.

We believe that the CUSUM method is ideal for the prospective analysis of the results of new procedures, because it is sufficiently sensitive, does not involve the use of complex software programs, and offers a statistical statement when a prescribed boundary line is crossed [2, 3, 16]. Unfortunately, CUSUM techniques have not, to our knowledge, been employed in other recent studies of learning curves of surgeons engaged in innovative procedures [1721]. In addition to its use in monitoring the results of individual surgeons, CUSUM methods can be employed at the institutional level, to determine whether a given unit is meeting its goals of optimizing postoperative patient outcomes.

Although this study and our previous ones focused on the use of CUSUM methods to analyze traditional cardiac surgical learning curves, recent advances in telementoring offer the promise to change learning curve paradigms. Our institution’s experience with telementoring using the Socrates System developed by Computer Motion (Santa Barbara, California) has to date included six procedures, including one general surgical, two pulmonary, and three cardiac cases. Advances such as these enable the "remote" surgeon to not only assist via videoconferencing technology, but also to manipulate robotic technology based in the operating room. These advances and the future advent of telesurgery offer the potential to favorably impact the learning curves of surgeons performing highly innovative procedures and to "flatten out" their CUSUM curves, especially during the initial learning phase.

Acknowledgments

This project was supported by a grant from the National Patient Safety Foundation. The authors acknowledge the assistance of Elizabeth J. Millar in manuscript preparation and George E. Moogk in figure construction. We also recognize the support of the Robarts Research Institute and the Lawson Health Research Institute.

References

  1. Mack M.J. Pro. Beating-heart surgery for coronary revascularization: is it the most important development since the introduction of the heart-lung machine?. Ann Thorac Surg 2000;70:1774-1778.[Free Full Text]
  2. Novick R.J., Stitt L.W. The learning curve of an academic cardiac surgeon: use of the CUSUM method. J Card Surg 1999;14:312-320.[Medline]
  3. Novick R.J., Fox S.A., Stitt L.W., et al. Cumulative sum failure analysis of a policy change from on pump to off pump coronary artery bypass grafting. Ann Thorac Surg 2001;72:S1016-S1021.[Abstract/Free Full Text]
  4. Altman D.G., Royston J.P. The hidden effect of time. Stat Med 1988;7:629-637.[Medline]
  5. McPherson K. Statistics: the problem of examining accumulating data more than once. N Engl J Med 1974;290:501-502.
  6. Williams S.M., Parry B.R., Schlup M.M.T. Quality control: an application of the CUSUM. Br Med J 1992;304:1359-1361.
  7. Society of Thoracic Surgeons’ Database. www.ctsnet.org, April 9, 2001.
  8. Weintraub W.S., Craver J.M., Jones E.L., et al. Improving cost and outcome of coronary surgery. Circulation 1998;98:II23-II28.
  9. Abramov D., Tamariz M.G., Fremes S.E., et al. Trends in coronary artery bypass surgery results: a recent, 9-year study. Ann Thorac Surg 2000;70:84-90.[Abstract/Free Full Text]
  10. Ascione R., Lloyd C.T., Underwood M.J., Gomes W.J., Angelini G.D. On-pump versus off-pump coronary revascularization: evaluation of renal function. Ann Thorac Surg 1999;68:493-498.[Abstract/Free Full Text]
  11. Ascione R., Lloyd C.T., Gomes W.J., Caputo M., Bryan A.J., Angelini G.D. Beating versus arrested heart revascularization: evaluation of myocardial function in a prospective randomized study. Eur J Cardiothorac Surg 1999;15:685-690.[Abstract/Free Full Text]
  12. Ascione R., Lloyd C.T., Underwood M.J., Lotto A.A., Pitsis A.A., Angelini G.D. Inflammatory response after coronary revascularization with or without cardiopulmonary bypass. Ann Thorac Surg 2000;69:1198-1204.[Abstract/Free Full Text]
  13. Ascione R., Lloyd C.T., Underwood M.J., Lotto A.A., Pitsis A.A., Angelini G.D. Economic outcome of off-pump coronary artery bypass surgery: a prospective randomized study. Ann Thorac Surg 1999;68:2237-2242.[Abstract/Free Full Text]
  14. Murkin J.M., Boyd W.D., Ganapathy S., Adams S.J., Peterson R.C. Beating heart surgery: why expect less central nervous system morbidity?. Ann Thorac Surg 1999;68:1498-1501.[Abstract/Free Full Text]
  15. Kshettry V.R., Flavin T.F., Emery R.W., Nicoloff D.M., Arom K.V., Petersen R.J. Does multivessel, off-pump coronary artery bypass reduce postoperative mortality?. Ann Thorac Surg 2000;69:1725-1731.[Abstract/Free Full Text]
  16. de Leval M.R., Francois K., Bull C., et al. Analysis of a cluster of surgical failures: application to a series of neonatal arterial switch operations. J Thorac Cardiovasc Surg 1994;107:914-924.[Abstract/Free Full Text]
  17. Watson D.I., Baigrie R.J., Jamieson G.G. A learning curve for laparoscopic fundoplication: definable, avoidable, or a waste of time?. Ann Surg 1996;224:198-203.[Medline]
  18. Liem M.S., van Stensel C.J., Boelhouwer R.U., et al. The learning curve for totally extraperitoneal laparoscopic inguinal hernia repair. Am J Surg 1996;171:281-285.[Medline]
  19. Blackwell K.E., Brown M.T., Gonzalez D. Overcoming the learning curve in microvascular head and neck reconstruction. Arch Otolaryngol Head Neck Surg 1997;123:1332-1335.
  20. Poulin E.C., Mamazza J. Laparoscopic splenectomy: lessons from the learning curve. Can J Surg 1998;41:28-36.[Medline]
  21. Higashihara E., Baba S., Nakagawa K., et al. Learning curve and conversion to open surgery in cases of laparoscopic adrenalectomy and nephrectomy. J Urol 1998;159:650-653.[Medline]



This article has been cited by other articles:


Home page
JRSMHome page
P. McCulloch
Developing appropriate methodology for the study of surgical techniques
J R Soc Med, February 1, 2009; 102(2): 51 - 55.
[Full Text] [PDF]


Home page
Card Surg AdultHome page
R. Jin and G. L. Grunkemeier
Statistical Treatment of Surgical Outcome Data
Card. Surg. Adult, January 1, 2008; 3(2008): 247 - 258.
[Full Text]


Home page
Qual Saf Health CareHome page
D. J Biau, M. Resche-Rigon, G. Godiris-Petit, R. S Nizard, and R. Porcher
Quality control of surgical and interventional procedures: a review of the CUSUM
Qual. Saf. Health Care, June 1, 2007; 16(3): 203 - 207.
[Abstract] [Full Text] [PDF]


Home page
Ann. Thorac. Surg.Home page
G. J. Murphy, C. A. Rogers, M. Caputo, and G. D. Angelini
Acquiring Proficiency in Off-Pump Surgery: Traversing the Learning Curve, Reproducibility, and Quality Control
Ann. Thorac. Surg., November 1, 2005; 80(5): 1965 - 1970.
[Abstract] [Full Text] [PDF]


Home page
J. Thorac. Cardiovasc. Surg.Home page
C. A. Rogers, B. C. Reeves, M. Caputo, J. S. Ganesh, R. S. Bonser, and G. D. Angelini
Control chart methods for monitoring cardiac surgical performance and their interpretation
J. Thorac. Cardiovasc. Surg., December 1, 2004; 128(6): 811 - 819.
[Full Text] [PDF]


Home page
J. Thorac. Cardiovasc. Surg.Home page
M. Caputo, B. C. Reeves, C. A. Rogers, R. Ascione, and G. D. Angelini
Monitoring the performance of residents during training in off-pump coronary surgery
J. Thorac. Cardiovasc. Surg., December 1, 2004; 128(6): 907 - 915.
[Abstract] [Full Text] [PDF]


Home page
Ann. Thorac. Surg.Home page
J. T. Reston, S. J. Tregear, and C. M. Turkelson
Meta-analysis of short-term and mid-term outcomes following off-pump coronary artery bypass grafting
Ann. Thorac. Surg., November 1, 2003; 76(5): 1510 - 1515.
[Abstract] [Full Text] [PDF]


Home page
Ann. Thorac. Surg.Home page
G. L. Grunkemeier, Y. X. Wu, and A. P. Furnary
Cumulative sum techniques for assessing surgical results
Ann. Thorac. Surg., September 1, 2003; 76(3): 663 - 667.
[Full Text] [PDF]


Home page
Ann. Thorac. Surg.Home page
J. M. Murkin and D. A. Stump
Introduction: outcomes 2001
Ann. Thorac. Surg., January 1, 2002; 73(1): S353 - 354.
[Full Text] [PDF]


This Article
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Personal Folders
Right arrow Download to citation manager
Right arrow Author home page(s):
Richard J. Novick
Bob B. Kiaii
W. Douglas Boyd
Right arrow Permission Requests
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Novick, R. J.
Right arrow Articles by Boyd, W. D.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Novick, R. J.
Right arrow Articles by Boyd, W. D.
Related Collections
Right arrow Coronary disease


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
ANN THORAC SURG ASIAN CARDIOVASC THORAC ANN EUR J CARDIOTHORAC SURG
J THORAC CARDIOVASC SURG ICVTS ALL CTSNet JOURNALS