Principles and a case study - Technology Assessment at Work, part 1

Continuous development and addition of new drugs, medical devices, and procedures have been characterized as a medical arms race that the health care community can no longer afford. Some technological advances in health care have made significant improvements in the health status of our population, some have not. In some instances, technology has been abused. That abuse has been fostered by the incentives of our health care system. Abuse may manifest itself in two ways:

* Premature adoption of technology without adequate evidence that it improves outcomes.

* Overuse of technology, regardless of its value, to provide a competitive market advantage or to generate revenue.

Health care systems, challenged by rising costs, have entered a period of rapid transformation. As part of this transformation, the way that decisions are made about technology is changing. In the past era of cost-based reimbursement, each new technology was a potential source of incremental revenue. In the future, health care organizations will compete for patients on the basis of value. Winners will be able to provide better clinical outcomes at lower costs than their competition.

Technology has the promise to improve outcomes, but it may, as it heretofore usually has, increase costs. Successful health care organizations will acquire the skill to identify and the courage to reject technologies that increase costs without commensurate improvement in outcomes. An effective process for technology evaluation will be essential.

Today, health care organizations use a variety of methods to assess and acquire technology. Some use review criteria that relate to the value of the technology with regard to outcomes. Reliable data for such benchmarking is scarce. Some compare new and emerging technology against existing technology, although, once again, outcomes data are rare. Most organizations buy what physicians want and use manufacturers and physicians as the primary source of technology information.

Unfortunately, most health care organizations do not look carefully enough at the value of a technology and compare its value to other technologies. Health care organizations frequently respond primarily to competitive, physician, and manufacturer pressure. This leads to a multitude of problems, including acquisition of inappropriate, excess, or unnecessary technology; distorted consumer and physician expectations; and increased costs. Such responses will be particularly problematic and painful in a capitated environment.

There are several key mistakes that occur during the diffusion of technology:

* Technologies are adopted simply because they are new.-- The impact of a new technology must be evaluated, paying attention to the process of care and to outcomes. This will include the technology's effect on costs, length of stay, and/or other services. This evaluation is best conducted through a predefined process with specific criteria that are applied to all requests for new technologies.

* Equipment is replaced solely because of its age.-- Technology replacement requests must be evaluated on the basis of existing equipment's ability to meet clinical need and on its safety and reliability. In addition, health care organizations must carefully match the capacity of existing technology with projected utilization in an aggressive managed care environment to avoid having too much technology. Long-range replacement plans are needed.

* Applications expand without adequate justification.-- Use of infusion pumps and telemetry are examples of technologies that expand within health care organizations without adequate clinical evidence that outcomes are improved or without cost-benefit analysis that proves savings.

* Numerous vendors are used and numerous models of products with essentially similar specifications are carried. -- Orthopedic implants and pacemakers are such devices. Reducing the number of brands and models used in the health care organization presents significant opportunity to reduce purchasing and inventory costs.

* Economies of scale in regionalizing technology decisions are missed.-- There are significant economies of scale in regionalizing technology decisions, including providing for the proper diffusion of technology throughout the continuum of care within a health care system, resulting in consolidation of acquisition and support tasks and their associated costs.

* Duplication of services.-- In a cost-conscious environment with scarce resources, duplication adds cost without value.

Technology Assessment at St. Luke's Hospital

Like politics, technology assessment is local. Information from national or even international sources is needed to evaluate a technology, but this information must be interpreted in the light of local circumstances. Technologies that might add value in a sophisticated urban market may not add real value in a rural setting. A highly specialized technology that adds value for a major referral center might increase costs without significant benefit if used in an organization with a less specialized focus.

The organizational process of technology assessment must be fair, scientifically credible, and effective. Deciding to adopt a new technology is usually a popular decision, but an organization with a fair, credible, and effective technology assessment process should also be able to win broad-based support for the rejection of a new technology.

Impartial people applying impartial standards will help physicians and others see that the local process is fair. At St. Luke's Hospital, a technology assessment committee with membership from both medical staff and administration is an effective forum. These decision makers must understand the clinical, technical, financial, and organizational issues related to new technology. They must be widely respected by peers for their knowledge and fairness. The process is best served by a group of senior physicians who represent a broad cross-section of the medical staff working with a group of senior administrators who are prepared to commit time to the process of technology assessment.

One committee member (usually a physician) is asked to research the scientific background and resource requirements of each new technology, to circulate a succinct summary to committee members, and to present this information verbally when the committee meets. This process does not require substantial staff resources.

Standards by which a technology can be judged include:

* The mission of the hospital. "Will this technology help us accomplish our mission?" In the immediate future, when technologies compete for limited resources, this question may be phrased as:

* "Will this technology help us accomplish our mission more effectively than other available technologies?" Framing decisions against the mission statement exposes arguments based on individual or departmental ego or "turf" and channels the discussion in directions most likely to help the organization fulfill its mission.

* "Assuming this technology works, will it help us accomplish our mission?" If the answer is no, there is no need for further action.

* Scientific evidence. If the technology promises to help the organization fulfill its mission, the group will ask, "Will this technology work?" Answering this question requires evaluation of scientific evidence. The best evidence will be randomized prospective studies that have followed patients for outcomes, including overall mortality and quality of life. Such evidence is hard to find for new technologies, and the group will often need to make a decision based on less powerful evidence, such as case-series reports. Least valuable will be single case reports and the claims of manufacturers.

In the absence of serious scientific evidence that the technology produces the claimed results, the group may simply decide to wait until such evidence is available. Regardless of the evidence available, criteria and indications for the technology's use must be established to measure outcomes and value.

Another problem the group will face is that, early in the evaluation of a new technology, there may be evidence that a technology achieves short-term clinical results but no evidence about long-term outcomes. One way this discrepancy can be addressed is by prospectively setting up indicators and measuring outcomes within the organization.

The group must then ask, "Will it work here?" Local experience with related technologies already in use in the hospital will be important. So will consideration of the likelihood that the organization and its members have the skills and experience needed to obtain results similar to those published and that the local patient population is similar to the one whose outcomes were published.

* Available resources. "What resources are required?" Next is consideration of the cost of the new technology. This will include not only capital and operating costs, but also the impact of the technology on existing services (both additional demands for support and replacement) and operational issues, such as space. Other considerations will include any applicable national standards or guidelines and the expectations of customers for high-quality care at a cost they can afford. Some technologies may be lifesaving therapies for which there is no current alternative and therefore must be acquired quickly. Other technologies may offer only marginal additional benefit, and their acquisition can be considered carefully and slowly and may be deferred.

At this point, a decision to adopt, reject, or defer a technology can be made. When a technology is formally adopted, implementation issues should be assigned to an ad hoc working group so that all affected departments know what they must do to make the technology work.

The technology assessment process will be credible only if the group is held accountable for its decisions. If a technology is adopted with the expectation that certain outcomes and costs will follow, outcomes and costs should be tracked through the organization's CQI system and should be reviewed by the group. CQI applied to technology assessment will help the group improve its decision-making process.

Recommendations to adopt a technology should include criteria for the use of the technology, credentialing criteria for clinical staff, and criteria to measure and modify outcomes.

A serious program of technology assessment requires institutional resources. At St. Luke's Hospital, the costs of the program have been largely related to the time and effort of selected medical staff. In addition, as a member of the Premier Health Alliance, the hospital has had additional support. Costs for a hospital using some outside resource for advice, database, and research support might reach $25,000 annually, in addition to time spent by members of the hospital's medical staff and administration. These costs will be more than balanced by savings from the deferral of a technology that does not meet the standards described in this article.

RELATED ARTICLE: A Short Glossary of Technology Assessment Terms

Technology: Device, pharmaceutical, product, procedure, or protocol.

Technology Assessment: Practical process of determining the value of a new or emerging technology in and of itself or against competing existing technologies using efficacy, effectiveness, outcome, risk management, strategic, financial, and competitive criteria. The purpose of technology assessment is fourfold: to determine appropriate technology consistent with the health care organization's mission, goals, and strategic plan; to determine the safety, efficacy, and cost-effective of a medical technology; to discriminate between appropriate and inappropriate indications for th e use of medical technology; and to devise quality-improvement methods to optimize the performance of the new technology.

Technology Planning: Systematic method of determining a health care organizations's technology-related needs and of setting short- and long-term priorities for overall technology needs based on strategic, financial, risk management, and clinical criteria.

Technology Acquisition: Rational process of determining which manufacturer provides the best equipment and support of a health care organization's needs.

Technology Management: Ensuring that technology in use in the health care organization is used properly and effectively and is properly supported.

RELATED ARTICLE: Summary of the Technology Assessment Process in a Hospital

Establishing a technology assessment committee

* Determine committee membership. -- Standing: Senior physicians, senior administrators, biostatisticians. -- Ad Hoc: Experts familiar with the technology and operational issues related to the technology to be discussed.

* Develop a mission statement of the committee.

* Develop a decision-making process requiring the committee to: -- Judge technologies against the organizational mission. -- Review and analyze scientific evidence of effectiveness and safety. -- Establish accountability for screening and selection of patients and for tracking outcomes.

Stage I: Preliminary Evaluation of a Technology

* Will this technology help the organization fulfill its mission?

* How much data are available? Are they rigorously tested?

* Is this technology based on reasonable scientific principals?

* What is the current level of diffusion (experimental, investigation, clinical trials, FDA-approved)?

Stage II: Detailed Evaluation

* Do accepted scientific principles underlie the technology?

* Is there scientific evidence that the technology works, and what is the quality of that evidence?

* What will the technology cost to acquire and operate? (Capital costs; operating costs; issues of staffing, training, space, maintenance, quality assessment and improvement, etc.)

* What will the impact of the technology be in fee-for-service and managed-care environments?

* Will the technology have an impact on existing services?

* Does the absence of existing alternatives make this acquisition urgent?

* Are there relevant national standards of care?

* Are there issues of patient or employee safety?

* What are the expectations of the customers of the organization?

Stage III: Implementation

If a recommendation to adopt the technology is planned:

* Who will help implement this technology (members of an ad hoc working group)?

* Can structural, process, and outcome quality improvement indicators be developed?

Stage IV: Recommendation

* Adopt.

* Defer until criteria are developed. Adopt with specific criteria indications for use.

* Defer until more information becomes available, costs are lower, or outcomes improve.

* Reject.

James Cowan, MD, is Chairman, Department of Medicine, St. Luke's Hospital, a tertiary are hospital, Bethlehem, Pa. He is chairman of the hospital's technology assessment committee.

David Berkowitz is National Accounts Manager, Varian Oncology Systems, Marietta, Ga. At the time this article was written, Mr. Berkowitz was Vice President of the Health Systems Group at ECRI, a not-for-profit health services research organization concentration on health care technology, Plymouth Meeting. Pa. The authors may be reached through Dr. Cowan at 801 Ostrum St., Bethlehem, Pa. 18105, 610/954-4642, FAX 610/954-4979.

COPYRIGHT 1996 American College of Physician Executives
COPYRIGHT 2004 Gale Group