| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Ann Thorac Surg 1997;64:1569-1573
© 1997 The Society of Thoracic Surgeons
American College of Surgeons, Chicago, Illinois
Abstract
Long-standing efforts to assess quality in medical care have been intensified by the striking expansion of managed care plans. Agencies such as the Joint Commission on Accreditation of Health Care Organizations and the National Committee on Quality Assurance have formalized the evaluation of health plan quality using criteria of structure, process, and outcome. A review of attempts to apply these criteria to individual physicians and to disease-specific interventions such as myocardial revascularization demonstrates the great difficulty of reliable quality assessment in this evolving surgical field. Cardiac surgeons must continue their work in deriving valid socioeconomic and clinical conclusions from The Society of Thoracic Surgeons and Veterans Affairs databases. This may prevent the precipitate adoption of newer treatment methods driven by entrepreneurial technology companies and large group purchasers of care. These entities tend to focus on economics rather than patient welfare. New technologies may also delude patients into insisting on treatment featuring short-term convenience and comfort despite less satisfactory long-term results. "Black box" methodology providing practice profiles and physician report cards must have critical validation.
How good is cardiac surgery today? An answer can be gained by considering three interrelated aspects: access, cost, and quality of care. One or another of these aspects tends to dominate sequentially the national discussion of surgical care; currently a major emphasis rests on quality of care.
This presentation reviews the quality of care in cardiac surgery. Specifically it addresses the operation of coronary revascularization by coronary artery bypass graft, a procedure commonly referred to by its acronym, CABG or "cabbage."
As myriads of these operations are carried out annually, immense effort has been expended by surgeons and others to improve the technique and results of the operation. Outcomes of the procedure have been used by extension to judge the quality of care by cardiac surgeons as well as the quality of the hospitals and the surgical units where the operations take place. Such derivative judgments are commonly unreliable.
Caveats notwithstanding, surgical results in myocardial revascularization are being used as an index of the quality of care. But the goal of asserting unequivocally to everyone's satisfaction that we are achieving a high-quality surgical outcome is elusive. This raises an important question: why has the pursuit of quality in American health care become such a dominant issue only a few years after the vigorous debate over a proposed health care plan aimed at universal access and containment of costs, with quality a secondary consideration? A large part of the reason for the shift of interest toward quality lies in the striking rise of managed-care plans.
Managed care has been described as payers telling physicians how to practice medicine. A major goal of managed care plans is cost containment. Financial success in such plans is partially dependent on spending as little money as practicable to care for enrollees in the plan. Although lowering costs does not necessarily decrease the quality of care, at some point restriction of expenditures will adversely affect medical care. Attempts to determine this critical point have moved the relationship between managed care and quality into its current prominence.
Interest in quality of care did not originate with the present expansion of managed care. From time immemorial physicians were confident, generally without nonprofessional challenge, that society relied on physicians' decisions to evaluate and maintain the quality of care. The conventional focus of the physician was on quality of care for the individual patient; this quality focus has now expanded to include groups of patients as well as the organizations that undertake to provide such group care. Individuals whose care is guaranteed by insurance arrangements are no longer called patients, but rather "covered lives." For a physician to think of his or her patients as economic units under an industrial contract involves not only a wrench in terminology but also an adjustment in attitude.
In view of the current intense interest in quality of care, a leading medical journal in late 1996 published a series of six articles on the subject [1–6]. The objective of these articles was concisely stated by David Blumenthal in the opening essay on "Quality of Care—What Is It?": "The purpose of the series is to review the major technical concepts and issues that are pertinent to current discussions about quality of care, to place those discussions in a political and social context, and to provide some guidance on how changes in techniques for measuring and improving quality may affect doctors and patients over the next decade" [1].
Blumenthal pointed out that quality of care definitions depend on the viewpoint of different groups involved in the medical care system. Physicians and other professionals focus on the appropriateness of care, the skill with which care is given, and the satisfaction of the patient with the result. In contrast, health care plans and so-called provider organizations focus on the accessibility of care and the distribution of resources. These entities may stress the need to restrict services in achieving budgetary goals. A third group, organized purchasers of care, evaluates quality by comparing one care plan with another, with emphasis on efficiency and cost. A satisfied, healthy patient is the common goal of physicians, provider organizations, and organized purchasers of care, but the criteria of quality in efforts toward that goal may vary widely among the three groups.
The diversity of groups interested in quality of care has made it hard to achieve consensus on definitions of quality. Donabedian's classic writings on the subject noted the importance of structure, process, and outcome [7–9]. Structural data concern such things as the characteristics of physicians and of the places where they work. Process refers to the way in which the encounter between patient and physician is conducted, including the history, physical examination, laboratory tests, and treatment. Outcome refers to the patient's status after the encounter. Is the patient better, the same, or worse, and who is making the evaluation?
In analyzing measurement for quality of care, Brook and others at the Rand Corporation have noted five methods of assessment [2]. The first three are implicit methods, that is, no specific criteria are set out in advance to define quality. Someone, usually a physician, reviews the medical record, first to see if care was "adequate." Second is an evaluation of whether an alternative method of care might have improved the outcome. Third, after review of the process and the outcome, is the care considered "acceptable"? A fourth method of assessment evaluates patient management under explicit process criteria set out in advance. And finally, we can compare the actual outcome with outcomes predicted under an established, validated model of care.
Physicians asked to define quality of care commonly do so in terms of process rather than outcome. For example, a patient with classic symptoms and signs of acute appendicitis might be considered as a recipient of low-quality care if treated without appendectomy. Such process analysis is also commonly used when comparing one physician or one health plan with others. But the condition we are examining here, namely CABG, may be an exception to the rule that process data are the preferred method of evaluation.
One reason for inadequacy of process data in evaluation of coronary bypass operations is the difficulty of measuring factors such as the surgeon's operative skill. On the face of it, an individual surgeon with the lowest operative mortality in a group might be thought to provide the highest quality of care, but this simplistic appraisal neglects important factors such as case selection and comorbidity. The method of selecting data for evaluation plays a critical role in the reliability of assessing quality of care.
Much of the information used in current assessments of quality comes from insurance records, which provide only negligible data related to evaluation of the result by the patient or physician. Such insurance data would provide little guidance to a new physician taking over care of the patient; rather, the new caretaker would need to study the hospital or office record of the patient. On the other hand, evaluation of a health care organization could not be reliably accomplished by review of an individual patient's chart. Moreover, giving high grades to an organization for its care of one set of diseases does not ensure that its care of other conditions would receive comparably high ratings.
The historical record of evaluating care by collection and display of reliable clinical data is too extensive to permit detailed review here. But even the most sketchy account must note the pioneering work of Dr Ernest A. Codman of Boston, as noted in the report of his Committee on Standardization of Hospitals, appointed under the auspices of the Clinical Congress of Surgeons of North America at its fourth annual meeting in November 1913 [10]. Codman wanted the efficiency of a hospital to be judged by the human results of its patient care. He thought that the skills of a hospital staff could be inferred from " ... the common sense notion that every hospital should follow every patient it treats, long enough to determine whether or not the treatment has been successful and then to inquire, if not, why not?"
Codman noted that hospitals could be compared by many standards, including cleanliness, nursing care, annual patient census, per capita cost, success in practice of its physicians and surgeons, and the quality of the physicians' scientific papers. Acceptable performance under all of these standards would not necessarily mean that the patient left the hospital cured of the presenting complaint.
Codman's reasonable but often unpopular policies were adopted and put into practice to some degree by the American College of Surgeons in the early years after its incorporation in 1912. Stephenson has noted that Codman's proposal for "standardization of hospitals" and American College of Surgeons founder Franklin H. Martin's plan for "standardization of surgeons" originated without prearrangement at the same 1912 meeting, "a portentous week for medicine in this country and throughout the world" [11]. Out of these efforts came the Hospital Standardization Program of the American College of Surgeons in 1918, supported exclusively by the College for more than 30 years. In 1951 the Joint Commission on Accreditation of Hospitals (JCAH) was formed under the auspices of the American College of Physicians, American College of Surgeons, American Hospital Association, and the American Medical Association. The Canadian Medical Association was an original participant but subsequently withdrew amiably to carry out similar functions after 1959 with the Canadian Commission on Hospital Accreditation.
Throughout its history, the JCAH has been resented and resisted for a variety of reasons, including invasion of privacy. However, in setting standards for hospitals JCAH has steadfastly insisted that it is not a policing body but rather an accrediting agency providing confidential appraisals to those hospitals that voluntarily seek accreditation. When the Federal government unilaterally adopted the judgments of the JCAH as a basis for ruling on the eligibility of hospitals to receive Federal funds, the JCAH involuntarily became a quasigovernmental body. If a hospital failed approval by JCAH, it was deemed ineligible for Medicare payments; thus the loss of "deemed status" could be damaging to a hospital's economic and educational survival. Such penalties could be waived by the federal government, and indeed, this was a regularly occurring, governmental subversion of the accrediting process.
For many years the relationship between a hospital and the JCAH was signified simply by the presence or absence of an accreditation certificate in the hospital lobby. As confidentiality gradually eroded and private corporations assumed an increasing role in influencing the medical care of their employees, hospital approval by an accrediting body took on increasing financial significance. Moreover, the rise of for-profit hospitals and their incorporation into ever-enlarging amalgamations was accompanied by heavy pressure on hospitals to increase efficiency and cut costs. There was also a growing tendency for purchasers of group care to demand some form of quality accreditation by an independent agency.
From its foundation in 1951 the JCAH operated for more than a quarter-century as the dominant arbiter of quality in hospital care. Its surveys targeted individual hospitals with only minimal attempts to survey health care networks and managed care plans. In 1979 a competitor appeared in the National Committee for Quality Assurance (NCQA), a private, not-for-profit entity formed by joint action of the main managed-care associations, the Group Health Association of America and the American Managed Care and Review Association [12]. The NCQA was formed as a private body to counter reported plans of the federal government to monitor health plans. In 1995 the founding organizations of NCQA merged to become the American Association of Health Plans. Five years earlier their sponsored agency, the NCQA, became independent with help from foundations and the federal government. Its independence from its originators was designed to counter the perennial charge that managed care plans skimp on care to save money, thereby putting quality of care at risk.
The NCQA developed the Health Plan Employer Data and Information Set (HEDIS) as a means of rating health care "providers." The initial emphasis was on measures related to utilization and management in health plans. HEDIS, version 2.5, has evolved into HEDIS 3.0, which the NCQA in its executive summary on the Internet describes as " ... a giant leap forward in the nation's effort to compare the performance of health plans ... " [13].
HEDIS 3.0 is oriented to outcome as well as process, suggesting that health plans will now be expected to measure how well patients are able to function in their daily lives. To quote again from the NCQA executive summary: "Not only is it important, for example, for a health plan to measure results in cases of coronary bypass graft surgery, but it is equally important that health plans demonstrate that they do what they can to keep members from using tobacco and, therefore, getting heart disease in the first place." HEDIS 3.0 has in its measurement objectives the full range of issues from prevention and early detection to acute and chronic care for patients of all ages and frequently occurring medical conditions.
From such wide-ranging "report cards" on health care plans it has been a logical step to focus the reports on individual practitioners. The federal Health Care Financing Administration began public disclosure of provider-specific data on treatment outcomes by publishing annual reports of mortality rates in hospitalized Medicare patients. The Health Care Financing Administration included all causes of hospitalization and based its reports on administrative data. The results were so clearly untrustworthy that the project was abandoned [14].
To avoid some of the methodologic problems of the Health Care Financing Administration study, the New York State Department of Health in 1991 developed the Cardiac Surgery Reporting System (CSRS). This system focused on CABG and went beyond mere administrative data to include clinical risk factors of proved significance in the prognosis of cardiac operations. By incorporating data on renal failure, congestive heart failure, chronic obstructive pulmonary disease, unstable angina, and the status of the left main coronary artery, CSRS became the first profiling system to provide a credible basis for comparing results of CABG among different surgeons. But there were a number of serious reservations about the accuracy and internal consistency of the CSRS data base, which cast doubt on the reliability of the New York State risk-adjustment model [15].
Green and Wintfeld [15] pointed out that the coding instrument was substantially revised during the years under study. Moreover, the surgeons whose work product was being evaluated could scarcely be unaware of this surveillance. Thus, the "sentinel effect" could move them to avoid operation on the sickest patients to improve their overall results. In addition, the five risk factors showed a substantial rise in incidence, suggesting that less strict criteria were used to indicate their presence. Comparison of such exaggerated risks with the results actually achieved would move the assessment in a favorable direction.
It is axiomatic that coordinators of formal clinical trials must prescribe training and monitoring sessions in each hospital to ensure accuracy and comparability of collected data among different institutions. The same conditions must obtain with outside assessment by governmental entities. When such precautions are taken, we may begin to approach the possibility of evaluating individual surgeons, which is currently not achievable.
The reported decline in risk-adjusted mortality in New York after the introduction of CSRS could be at least partially accounted for by a recent national decline in mortality rates after myocardial revascularization in elderly patients under Medicare [16]. Multiple factors influence such a general improvement in operative results; years may pass before their cumulative effect is fully manifest.
For example, the technical modification of bypass grafting on the beating heart, introduced at least a quarter century ago [17], has recently begun to enjoy wider clinical application [18]. There is aggressive marketing of this technique by the company making specialized equipment to facilitate such operations. At the same time, another company is pushing a kit of instruments to facilitate open cardiac procedures with the heart arrested. Both companies teach doctors to use their methods in sessions lasting only a few days. Certain large, for-profit hospital corporations are providing training programs for surgeons in open heart programs at their own chain of hospitals. In this unseemly rush to promote competing technologies by commercial manufacturers and for-profit hospital networks, the question of patient safety is hardly receiving adequate emphasis.
It is obvious that these newer treatment modalities are attractive to patients, who are understandably eager for procedures that minimize discomfort and assure their early return to full function. Employers who pay the bills are also interested in rapid restoration of an employee to full productivity with the least cost for an episode of illness. But such a focus on cost and efficiency of care should be balanced by an unflinching assessment of safety and effectiveness. To do this in the current feverish marketing environment is difficult indeed. And it is apparent that adoption of a quality standard is impossible when so many factors affecting cardiac surgery are in constant flux.
As the criteria for judging quality become increasingly sophisticated, there is a corresponding increase in the difficulty of collecting elaborate clinical data and using it in ways that have a positive effect on everyday practice. Computer software makes analysis of vast data sets relatively simple, but the reliability of the methods that generate these appraisals and comparisons must be regularly validated [19]. Stated another way, the "black box" methodology that gives us practice profiles and physician "report cards" cannot be accepted without question [20].
Comment
Quality has always been an important component in the tripartite relationship with access and cost of medical care. With the spread of managed care, the large industrial purchasers of care have increasingly demanded defined standards of quality among the organized health plans that provide care for their employees. This has encouraged the growth of independent assessment organizations, separate from the health care plans that contract to provide defined services to target populations or "covered lives."
Among assessment organizations the NCQA has come to occupy a dominant role in evaluating health care plans. Its HEDIS has achieved wide acceptance by large industrial purchasers of care for their employees. Most of the measures in HEDIS concern management issues rather than focusing on care for individual patients. As HEDIS is gradually updated to add more measures of performance by individual physicians, the reliability of its assessment will be increasingly challenged. New York State since 1989 has compiled annual statistics on risk-adjusted mortality for CABG related to specific hospitals and surgeons. These "report cards" have been associated in successive years with apparent improvement in mortality after coronary artery bypass grafts. However, careful analysis of the data casts doubt on the conclusion that genuine improvement in care has occurred.
In addition to unreliability of the data and national improvement in results of care for Medicare patients independent of measures taken in New York State, we are currently in a technologic revolution affecting the nature of myocardial revascularization procedures. A major impetus driving the extension of limited-access bypass operations as well as coronary operations on the beating heart comes from commercial firms selling equipment to facilitate one or another of these new techniques. Hospital chains are equally eager to promote these newer modalities of treatment, lest they lose market share in this highly competitive field.
This competition is no longer a subject for discussion only in professional meetings and hospital board rooms. On April 22, 1997, the Wall Street Journal ran a front-page story about newer developments in open heart surgery, which it characterized as a "$30 billion-a-year business." The article noted that some heart surgeons saw these newer, more patient-friendly approaches as " ... an opportunity to recapture patients lost by the tens of thousands to angioplasty... ." The author, Ron Winslow, spoke to the " ... thin line between pioneering innovation and rash experimentation" while recognizing the role of the informed patient in calling for an operative approach that promised a shorter, less disabling postoperative course. So far the number of patients treated by these newer methods is only a few thousand, compared with millions managed over the years by conventional operations. With the hundreds of cardiovascular surgical teams poised to employ these new approaches, the curve of use is certain to rise steeply.
It should not be assumed that the surgical profession has been idle in addressing these socioeconomic issues. Beginning in 1995 with a symposium on Risk Management in CABG at the annual meeting of The Society of Thoracic Surgeons, there have been two subsequent symposia in 1996 and 1997, with the 1996 proceedings published as a Supplement to The Annals of Thoracic Surgery [21]. The papers covered the databases of The Society of Thoracic Surgeons and the Department of Veterans Affairs, the activities of the Health Care Financing Administration in evaluating quality/performance measurement, along with reimbursement in the United States and Canada. Other presentations dealt with retained graft patency after revascularization and the factors that determine financial risk under managed care. The most recent symposium in this field occurred on February 1, 1997, at the annual meeting of The Society of Thoracic Surgeons; the six papers and discussions will also appear as a supplement to The Annals of Thoracic Surgery.
Summary
Of the three components vital in evaluating our system of medical care, quality is currently receiving equal or greater attention than cost or accessibility. The nature of managed care plans and the demands of large, group purchasers of care have focused attention on quality.
Industrial quality control methodology to evaluate the performance of health care plans has been widely adopted. The JCAH and the NCQA are the leading national bodies devoted to assessment of quality in organizations providing medical care. They use the classic Donabedian triad of structure, process, and outcome as the criteria for rating hundreds of hospitals and hospital networks.
The HEDIS is the evolving instrument used by NCQA in rating health care plans. The current version of this instrument, HEDIS 3.0, has now included more measures of outcome, but the overall applicability of the evaluative mechanism is still slanted toward structure and process in health care plans. "Report cards" on surgeons in New York State and elsewhere have been characterized by serious deficiencies, including data that are unreliable or missing. Comparisons of surgical performance over successive years suffer from deliberate institutional upgrading in the criteria of surgical risk, such as overemphasis on the degree of cardiac, renal, or pulmonary malfunction. This results in spurious evidence of improved results because the preoperative assessment has been unduly pessimistic in positing risk.
In addition to these institutional manipulations of the data to bias the results, we are currently seeing a radical revision in technical approaches to myocardial revascularization. "Minimal access" operations, with CABG using expensive patented equipment to facilitate the bypass, or limited grafting on the beating heart are the current fashions. Although the overall safety and long-term results of these approaches are still uncertain, they are being vigorously promoted by equipment manufacturers and by group purchasers of care for economic reasons. Patients are likewise attracted to these novel methods because they promise less discomfort and a shortened period of disability.
Whatever may be the ultimate evaluation of safety and efficacy of these newer approaches to myocardial revascularization, it is clearly impossible to make year-to-year comparisons of an individual surgeon's performance whe the field is evolving so rapidly. If the CSRS in New York was plagued by flaws in design and difficulty in application, there will be even greater problems in achieving reliable comparisons of quality performance when the operative approaches are in flux from year to year. Moreover, the "black box" methodology used to provide practice profiles and physician report cards must also be subjected to ongoing, critical appraisal.
Footnotes
Presented at Cardiovascular Surgery—Then and Now, University of Virginia Medical Center, Charlottesville, VA, April 26, 1997.
Address reprint requests to Dr Hanlon, American College of Surgeons, 55 E Erie St, Chicago, IL 60611 (e-mail: rhanlon{at}facs.org).
References
This article has been cited by other articles:
|
|
 |
|
  E. A. Rider and J. M. Perrin Performance Profiles: The Influence of Patient Satisfaction Data on Physicians' Practice Pediatrics, May 1, 2002; 109(5): 752 - 757. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
