Ann Thorac Surg 2007;83:179-184
© 2007 The Society of Thoracic Surgeons
Original Articles: Cardiovascular
Measurement of Technical Performance in Congenital Heart Surgery: A Pilot Study
Luis Alesandro Larrazabal, MDa,
Pedro J. del Nido, MDa,
Kathy J. Jenkins, MD, MPHb,
Kimberlee Gauvreau, ScDb,
Ronald Lacro, MDb,
Steve D. Colan, MDb,
Frank Pigula, MDa,
Oscar J. Benavidez, MDb,
Francis Fynn-Thompson, MDa,
John E. Mayer, Jr, MDa,
Emile A. Bacha, MDa,*
a Department of Cardiac Surgery, Children’s Hospital Boston, Harvard Medical School, Boston Massachusetts
b Department of Cardiology, Children’s Hospital Boston, Harvard Medical School, Boston Massachusetts
Accepted for publication July 18, 2006.
* Address correspondence to Dr Bacha, Cardiac Surgery, Children’s Hospital Boston 300 Longwood Ave, Boston, MA 02115 (Email: emile.bacha{at}cardio.chboston.org).
 |
Abstract
|
|---|
BACKGROUND: Although adequacy of repair after congenital heart surgery is a crucial determinant of clinical outcome, there is no current method of assessment. We sought to develop a process to measure the adequacy of repair for a diverse group of congenital heart procedures.
METHODS: Selected surgical procedures, consisting of repair of ventricular septal defect (VSD), tetralogy of Fallot (TOF), complete common atrioventricular canal (CAVC), and arterial switch operation, were divided into component subprocedures, each of which was assessed separately. Three outcome categories of "optimal," "adequate," and "inadequate" were defined by consensus according to postprocedure echocardiographic assessment. Outcome categories for conduction disturbance were also created. All patients undergoing one of the four procedures in 2004 were identified, and each subprocedure was assessed. Other clinical data were obtained from medical records. Repairs were scored as "optimal" if all attempted subprocedures and conduction were optimal, and "inadequate" if any was inadequate.
RESULTS: A total of 138 procedures were included. VSD repair was done in 46 patients (33%), TOF repair in 33 (24%), arterial switch operation in 36 (26%), and CAVC repair in 23 (17%). Optimal technical score was found in 28 (20%), adequate in 106 (77%), and inadequate in 4 (3%) (2 VSD, 1 TOF, 1 CAVC). Median length of stay was 8 days, and no patients died.
CONCLUSIONS: Despite procedural diversity and complexity, technical adequacy of repair can be assessed for congenital heart surgery.
|
Introduction
|
|---|
There has been increasing public interest in monitoring the quality of cardiac surgical performance. Surgeon-specific mortality for coronary artery bypass grafting has been published in the New York Times. The publication of the Bristol Royal Infirmary Inquiry Panel report [1] led to the introduction of clinical governance strategies into the United Kingdom National Health Service.
Surgical performance is usually indirectly measured by postoperative outcome of the initial hospital stay by means of risk-adjusted audits [2–4]. Although risk-adjustment is important to assess performance and compare outcomes amongst individuals or institutions [5], statistical inferences alone cannot be used to determine what is considered acceptable performance [6, 7]. Today’s available methods look at the "big picture," from diagnosis to surgery and postoperative care. It is somewhat deceiving, however, to judge an individual surgeon’s performance by using postoperative outcome data such as 30-day survival or hospital survival. A poor outcome can be the result of a technical error, a nursing mistake, a drug error, or substandard intensive care monitoring.
Intraoperative technical performance is one of the most important, if not the most important, parts of the therapeutic process and determines postoperative outcomes. None of the currently available quality monitoring tools measures technical performance per se [8–11]. The purpose of this study was to pilot the development of a tool that is easily applicable, valid, and reliable to measure the technical performance of congenital heart surgeons.
|
Material and Methods
|
|---|
The Children’s Hospital Boston Internal Review Board approved this study. A waiver of informed consent was obtained. Patient data was anonymized in our database in compliance with the hospital requirements.
Creation of the Tool
Four common surgical procedures—ventricular septal defect (VSD) repair, tetralogy of Fallot (TOF) repair, arterial switch operation (ASO), and complete atrioventricular canal (CAVC) defect repair—were selected for this pilot study. They were subsequently subdivided into individual components or "subprocedures" according to the specific anatomic region of the repair; for example, TOF repair was subdivided into an atrial level, a ventricular level, a right ventricular outflow tract, and a pulmonary artery repair.
A technical scoring tool was then created that defined three possible categories for each subprocedure: "optimal," "adequate" and "inadequate" (see Appendices 1 to 4). To score the categories, echocardiographic parameters were decided by consensus of a group of cardiologists and cardiac surgeons. An "overall outcome" was also created for the main surgical procedure by assigning an optimal score if all subprocedures were graded as optimal, an inadequate score if any of the subprocedures was graded as inadequate, and an adequate score for everything in between. Regions with limited postoperative echocardiographic yield, such as the aortic arch, were not scored. Similarly, procedures with debatable clinical indications, such as subaortic membrane surgery, were not included. Only anatomic regions that were operated on were scored (see Appendices 1–4).
Data Collection
A retrospective review was conducted of all patients undergoing VSD repair, TOF repair, ASO, and CAVC repair during the calendar year 2004 at Children’s Hospital Boston. Discharge echocardiogram reports were reviewed. Patients who did not have a discharge echocardiogram were excluded along with those who had anatomic areas corrected that were not scored within this study (eg, ASO with aortic arch repair).
Other data collection included preoperative patient characteristics such as age and weight at surgery, noncardiac anomalies, procedural characteristics such as cardiopulmonary bypass complications, myocardial protection problems, bleeding, and procedure times (cardiopulmonary bypass and myocardial ischemia time), and postoperative patient characteristics such as significant change in ventricular function from preoperative state, conduction system disturbances, length of stay, and in-hospital mortality.
Data Analysis
For each of the four procedures studied, technical scores were tabulated both overall and for each of the component subprocedures. For subprocedures that were included as part of more than one surgical procedure (eg, ASD repair can be a component of VSD repair, TOF repair, and ASO), technical scores were combined and tabulated across all cases of the subprocedure. The Fisher exact test was used to explore the relationships between the technical score and outcomes, patient characteristics, or surgeon.
|
Results
|
|---|
We identified 177 surgical procedures, of which 39 were excluded (VSD 21, TOF 4, ASO 8 and CAVC 6) because 35 had either an excluding anatomic variation (eg, Ebstein malformation of the tricuspid valve, Taussig-Bing malformation with hypoplastic left ventricle, swiss cheese septum, pulmonary atresia, etc) or procedure (eg, aortic arch repair) or both, and four did not have a discharge echocardiogram (all VSDs). Included were 138 patients representing 20% of all on-pump procedures performed in 2004 at Children’s Hospital Boston.
Procedures were performed by four attending surgeons, each performing 42%, 33%, 17% and 9% of the studied procedures, respectively. The median patient weight was 4.2 kg (range, 1.3 to 60.5), and the median age was 92 days (range, 1 to 9670 days). Seventeen (12%) had noncardiac structural anomalies, and 8 (6%) were born premature. The discharge echocardiogram was performed at a median of 4 days (range, 1 to 22 days) after surgery, and the median length of stay was 8 days (range, 3 to 46). There were no deaths. One patient required surgical reintervention for a residual ventricular septal defect after repair of TOF. Procedures consisted of VSD repairs in 46 (33%) TOF repairs in 33 (24%), arterial switch operations in 36 (26%), and CAVC repairs in 23 (17%).
Table 1
shows the overall scoring outcome for each surgical procedure. Optimal score was found in 28 (20%) of cases, adequate in 106 (77%), and inadequate in 4 (3%) (3 residual VSD >3 mm, 1 left AV valve regurgitation >4 mm). Tables 2 through 5
summarize the scoring of the subprocedures within each of the four surgical procedures, and Table 6
presents the scores for each subprocedure scored independently of the operation during which it was performed.
Looking at each of the four surgical procedures separately, we identified no associations between technical score and length of stay, surgeon or total pump time, or with demographic variables such as prematurity, Down syndrome, or major noncardiac structural anomaly. A similar examination of individual subprocedures found no associations.
|
Comment
|
|---|
A comprehensive, risk-adjusted method to assess quality and outcomes in congenital heart surgery remains elusive. Measuring performance in congenital heart surgery has been marked by the use of surrogate end points that usually involve outcomes measures such as 30-day mortality or hospital mortality [7]. Observed occurrence of outcome measures as a proportion of cases performed is often compared with expected performance by using large databases such as the Society of Thoracic Surgeons Congenital Heart Surgery Database. Other methods include cumulative sum charts or variable life-adjusted display charts [8–11]. All of these methods use as a baseline the measures observed and expected clinical outcomes; however, does worse then expected mortality mean that the surgeon is technically inadequate? A case in point is the analysis of the Bristol affair, where surgeons were initially accused of being solely responsible for poor outcomes but were eventually partially exonerated after in-depth review of records by a panel of experts [12]. Responsibility was assigned to the entire care team. In almost all cases, the problem ended up being the system and not the individual surgeons operating at the sharp end of the process.
Compared with adult cardiac surgery, for example, pediatric cardiac surgery is characterized by smaller total volumes of cases and a much greater variety of operations. This makes statistical inference of technical performance more difficult, because it is typically large sample sizes of a few operations (eg, CABG) that allow technical performance to be deciphered from other covariates. The present study represents an attempt at partially addressing this problem by subdividing complex procedures into basic components. Each component of a given repair can then be scrutinized and analyzed separately. The technical performance of a given surgeon with VSD closure, for example, is not just studied by looking at the technical score for isolated VSD procedures; rather, all VSDs can be included in the analysis, thus increasing the statistical yield.
For this pilot study, we focused on standard cases with the least possible echocardiographic or clinical controversy. Thus, we chose four mainstream pediatric cardiac operations. The technical score was not associated with surgeon, length of stay, or any of the other variables studied. This is likely because 77% of cases ended up with an "adequate" score. This uneven distribution resulted in insufficient variability in the scores. Validation of this concept will thus require further study and more data collection, presently in action.
Our study should also be viewed in light of several limitations. The main limitation is that it is based on echocardiography, a notoriously operator-dependent technique with well-known limitations, including the assessment of aortic arch gradients, aortopulmonary shunt patency, and the dependency on thoracic windows. Conversely, for a scoring system to be widely applicable, simplicity is a key factor. Echocardiography fulfills this criterion because it is a bedside test that is widely available and has proven accuracy as a stand-alone test in the preoperative and postoperative evaluation in congenital heart surgery [13]. Basing a technical measurement tool on invasive testing such as angiograms or magnetic resonance imaging would be impractical, costly, and make little sense. Nevertheless, it would be conceivable to add other modalities, such as blood pressure gradients for aortic arch surgery, into a comprehensive technical score.
The criteria used to define the three scoring groups were chosen empirically. We broadly defined "optimal" as the perfect result and "inadequate" as a technical result that either leaves a need for reintervention or leaves at least moderate residual disease, which presumably places the patient at increased risk for symptoms or reintervention, or both, at a later time. We do not advocate that the scores be adopted as final. Rather, once our hypothesis is tested, future revisions will be planned as needed. For this attempt, we included only mainstream cases and tried to avoid controversial subjects such as defining scores for subaortic membrane resection, aortic arch surgery, or ventricular function.
One further unaddressed problem is risk adjustment. Should a technical score include patient-specific nonmodifiable risk factors such as weight or prematurity, as is typically done in clinical performance risk-adjustment methods such as the Risk Adjusted Classification for Congenital Heart Surgery (RACHS-1) or Aristotle Performance Index [4, 6], or should it include only technique-specific anatomic risk factors such as a single papillary muscle in CAVC or anomalous coronaries in ASO?
Finally, surgical excellence, beyond excellent three-dimensional perception and manual dexterity (presumably captured in the technical score), also involves less measurable aspects such as the ability to prevent or deal with complications, understanding of cardiovascular physiology, cardiopulmonary bypass and myocardial protection, as well as "human factors" components such as situational awareness, decision-making skills, multi-tasking, and prioritizing, all of which are not captured with this tool.
In summary, as part of a quality improvement evaluation process, we present the concept of a technical score for congenital heart surgery, with results for four common operations. We conclude that despite procedural diversity and complexity, adequacy of repair can be assessed for congenital heart surgery. Further study and data capture will be required to validate this score.
|
Appendix 1
|
|---|
|
Appendix 2
|
|---|
|
Appendix 3
|
|---|
|
Appendix 4
|
|---|
|
References
|
|---|
- Murphy JF. The Bristol Royal Infirmary Inquiry 18th July 2001 Ir Med J 2001;94:228.[Medline]
- Lacour-Gayet F. Risk stratification theme for congenital heart surgery Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu 2002;5:148-152.[Medline]
- Jenkins KJ, Gauvreau K, Newburger JW, Spray TL, Moller JH, Iezzoni LI. Consensus-based method for risk adjustment for surgery for congenital heart disease J Thorac Cardiovasc Surg 2002;123:110-118.[Abstract/Free Full Text]
- Jenkins KJ. Risk adjustment for congenital heart surgery: the RACHS-1 method Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu 2004;7:180-184.[Medline]
- Kang N, Cole T, Tsang V, Elliott M, de Leval M. Risk stratification in paediatric open-heart surgery Eur J Cardiothorac Surg 2004;26:3-11.[Abstract/Free Full Text]
- Lacour-Gayet F, Jacobs JP, Clarke DR, et al. Performance of surgery for congenital heart disease: shall we wait a generation or look for different statistics? J Thorac Cardiovasc Surg 2005;130:234-235.[Free Full Text]
- Stark JF, Gallivan S, Davis K, et al. Assessment of mortality rates for congenital heart defects and surgeons’ performance Ann Thorac Surg 2001;72:169-174discussion 74–5.[Abstract/Free Full Text]
- Blackstone EH. Monitoring surgical performance J Thorac Cardiovasc Surg 2004;128:807-810.[Free Full Text]
- Rogers CA, Reeves BC, Caputo M, Ganesh JS, Bonser RS, Angelini GD. Control chart methods for monitoring cardiac surgical performance and their interpretation J Thorac Cardiovasc Surg 2004;128:811-819.[Free Full Text]
- Treasure T, Gallivan S, Sherlaw-Johnson C. Monitoring cardiac surgical performance: a commentary J Thorac Cardiovasc Surg 2004;128:823-825.[Free Full Text]
- Spiegelhalter DJ. Monitoring clinical performance: a commentary J Thorac Cardiovasc Surg 2004;128:820-822.[Free Full Text]
- Walshe K, Offen N. A very public failure: lessons for quality improvement in healthcare organisations from the Bristol Royal Infirmary Qual Health Care 2001;10:250-256.[Free Full Text]
- Tworetzky W, McElhinney DB, Brook MM, Reddy VM, Hanley FL, Silverman NH. Echocardiographic diagnosis alone for the complete repair of major congenital heart defects J Am Coll Cardiol 1999;33:228-233.[Abstract/Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
E. A. Bacha, L. A. Larrazabal, F. A. Pigula, K. Gauvreau, K. J. Jenkins, S. D. Colan, F. Fynn-Thompson, J. E. Mayer Jr., and P. J. del Nido
Measurement of technical performance in surgery for congenital heart disease: The stage I Norwood procedure
J. Thorac. Cardiovasc. Surg.,
October 1, 2008;
136(4):
993 - 997.
[Abstract]
[Full Text]
[PDF]
|
|
|
Top
Abstract
Introduction
