Matching Treatment to Patient Needs in Opioid Substitution Therapy
Treatment Improvement Protocol (TIP) Series 20

Appendix C -- Methodological Approaches to Cost-Efficiency and Cost-Effectiveness Analyses ¹

This appendix provides a conceptual framework for conducting cost-effectiveness analysis. It uses some innovative methodologies that have not been used to date in the substance abuse treatment field. The methodological approaches focus on five specific concerns or issues:

• What is the most cost-effective way to implement a program? An intervention?
• Among several programs performing in a most cost-effective way, which is the most efficient and effective program intervention?
• What is the best way to index efficiency and effectiveness when measures involve multiple outcomes and multiple cost components?
• How does one integrate a variety of different cost-effective measures into a program that maximizes patient benefits and minimizes cost over the entire period of treatment?
• What is the most productive scope of program intervention on a societal level? How large should the program be in relation to its overall societal benefit?

In the following sections, these methodological areas are briefly discussed and some examples of the applications of these methods are provided.

The first step in any cost-effectiveness analysis involves ascertaining which is the optimal or most efficient and effective implementation of the program. Without ascertaining the most cost effective implementation, it is impossible to make a comparison between multiple programs. For example, comparing a well-designed program that is poorly implemented with another program that is perhaps less cost effective but well managed may produce results contrary to expectations.

To develop this notion, utilization review concepts used by healthcare management personnel are employed, as well as average cost function methods used by economists. The intent is to ascertain at what point a program, as it is administered over time, achieves the maximum benefit to the patient, per unit cost.

In microeconomics, this maximum benefit is analogous to the average cost curve where program efficiency is viewed as a point where production is at such a level that marginal cost equals average cost.

The approach discussed here differs from the traditional economic one in that the product or the output (that is, the effectiveness of the program) is allowed to vary over time; that is, changes in the functional status of the treated population in a particular program may vary as the program is being administered.

The utilization review methodology ascertains the point in patient treatment when the marginal gain in program or individual goal is no longer justified by the cost of the program. The combination of the two concepts of utilization review and average cost function allows for the determination of the optimal implementation for each program. For example, if counseling visits beyond ten, on average, for patients in methadone treatment programs are found to bring no additional benefits (that is, additional gains in level of functioning), can we justify continuation of counseling treatment?

One approach to developing this methodology further is to use an algorithm that can be adapted to a variety of program modalities. A standardized package of instruments for measuring outcomes and cost, as well as procedures for calculating the optimal implementation point would be disseminated and tested. The methodology could be refined by introducing client characteristics into the algorithm equations in order to control for case mix differences across programs. Given the fact that resources are scarce and demand continues to increase, it is essential to target resources efficiently and effectively. One way to do this is to use a rationale for providing a certain number of treatment sessions based on some optimum level of cost and outcome.

Once the most cost-effective implementation of each program is established, it is possible to ascertain which program intervention is most efficient or effective according to established criteria. This methodology is the most common application of the cost-effectiveness literature wherein a variety of programs are examined and the most cost-effective one is selected. As discussed above, unless each program is implemented in a most cost-efficient way, such comparisons are likely to be meaningless.

For example, consider the comparison of a minimal versus moderate level of counseling support for two types of methadone programs. In the first program, methadone treatment is accompanied by weekly counseling sessions, and in the second, there is additional medical care, career counseling, and family support groups. The comparison between the two programs in terms of which is the most cost effective should first be based on the optimal implementation of each program. At that point the application of standard cost-benefit analysis involving either benefit-to-cost ratios or the difference between benefits and cost can be used to choose the most cost-effective program.

This implementation of the cost-effectiveness methodology will, however, require allowance for different client groups and population attributes. For example, the minimal methadone program implementation may be most suitable for a population which is gainfully employed, whereas the more intensive or moderate program will be more appropriate for a population that is unemployed. Cost effectiveness should always be considered with reference to its appropriateness for particular population groups. In the first case, one should always seek to implement the most cost-effective protocol of the program. Otherwise, its potential cannot truly be realized. In the second case, the issue is which program gives the most per dollar spent and at the same time is the most appropriate for a particular population group.

The measurement of efficiency, in standard economic applications, uses production economics to determine optimal output. In these types of applications there is an assumed direct mapping between the resources utilized and the output and outcomes of the resources. In the social and health science field, the production function which maps the intervention and the outcome is much more complicated.

Our approach to measuring and indexing efficiency differs from conventional economic methods in that it differentiates between efficiency and effectiveness. In the first case, efficiency is measured by ascertaining the degree to which a program or the implementation of a program for a particular client achieves its maximum efficiency. We do this by comparing resource utilization per output with reference to programs providing similar activities.

In the second case, we establish a reference frontier which reflects the best practice in the conversion of activities to outcomes. Here we measure effectiveness. The distinction made between efficiency and effectiveness enables us to ascertain a relationship between the two.

A number of previous studies (Schinnar et al., 1990, 1993, 1993a)² have demonstrated that the relationship between efficiency and effectiveness is curvilinear in that minimum effectiveness is observed at both minimum and maximum levels of efficiency, and maximum effectiveness is found at a moderate or middle range of efficiency. This measurement suggests that some moderate level of efficiency and effectiveness may be the one that is most optimal for program implementation.

Another important aspect of the measurement of efficiency and effectiveness is the indexing procedure. As demonstrated in a variety of studies in the area of substance abuse, the measurement of outcomes requires multiple indicators. Similarly, the measurement of cost also involves multiple dimensions such as the direct cost of program provision; the cost to the client for services such as childcare, transportation; lost employment opportunity, and so forth; and the societal benefit that occurs from reductions in crime and costly medical utilization.

Similarly, program activities involve many interventions such as counseling, medical care, and vocational assistance. How does one combine these multiple measures into a single index? Today, most methodologies involve the development of multiple ratios of inputs to outputs or outcomes to costs, followed by applying ad hoc procedures to obtain aggregations of these indexes. We suggest the use of a novel methodology known as Best Practice Frontier Analysis to develop measures of efficiency and effectiveness to effect such aggregations (Schinnar, 1980; Charnes et al., 1978)³. The method has been tested and used for the past 15 years in numerous applications and enables the derivation of distinct indexes for efficiency and effectiveness involving measures and scales spanning multiple dimensions. The method also circumvents the problem of the need to convert such scales into commensurate dimensions as is often done in cost calculations used in traditional cost-benefit analysis. In this manner, cost can be measured in dollars or in time lost or in a variety of other ways (i.e., family burden) without the need to convert these into monetary values which, at times, trivialize the true meaning of the measure itself.

In addition to the measurement of efficiency and effectiveness, multivariate regression methodology can be used to test hypotheses regarding causes for efficiency and effectiveness, as well as to ascertain what organizational determinants, client case mix factors, or funding mechanisms, and so forth are discretionary in affecting the performance of the program. The index of program efficiency and effectiveness is used as the dependent variable and regressed on a list of client, provider and organizational/program characteristics.

Drug abuse is viewed as a chronic disease that requires long-term treatment, occasionally continuously and at other times intermittently. If one considers an episode of illness and the treatment during that episode, one must analyze the progression of the patient through various stages of the illness or disease. Our objective in exploring these issues is to develop the appropriate methodology to explain or model this process and to determine the optimal or most cost-effective assignment of programs to clients at various stages of their illness.

Recognizing that there is considerable uncertainty associated with the effectiveness of the program and the transition between one stage of illness and another, we propose the application of dynamic programming techniques (Bellman and Dreyfus, 1962)⁴ to develop the most cost-effective assignment or choice of program for various stages of the client's treatment progression. The methodology partitions the illness and the stages of treatment into various milestones determined by a panel of experts. The next step is to identify the probabilities of transition between the various stages and list the optimal programs available for treatment in these stages. An objective function maximizes, over time, the benefit-to-cost ratio of the administration of the program and selects for each stage the most optimal program to be administered in order to produce the most cost-effective sequence of programs throughout the entire course of the treatment. Such dynamic programming protocols could also be developed for different types of patients based on various attributes.

The key to this methodology is that, unlike traditional cost-benefit analysis, which looks at each stage for particular client populations and chooses the optimum strategy, this strategy allows us to look at the entire treatment process rather than each individual episode. The utility of this particular approach would be at a systems or policy level rather than the individual program level where the question pertains to specific program design implementation. In this case, the question pertains to developing a complete system of programs that coordinates the sequencing of patients between these programs in a way that would maximize their benefit from the treatment.

In this methodological exploration, we move from the clinical outcome domain of cost-effectiveness to address a broader societal question. The question to be addressed is what is the most cost-effective level of treatment for the population as a whole as compared to the benefits from the treatment to society.

For example, should we treat only the most severely ill patient, should we treat those who are employed, should we treat those who are unemployed, and should we treat those who are moderate drug abusers with the chance that they may deteriorate? Clearly, with each of these populations and severity of drug abuse, there is an associated program cost as well as a social opportunity cost. Our objective is to extrapolate from the microanalysis level of specific programs the point of intervention where the marginal gains to society from further drug treatment are offset by the increasing cost of program administration.

A methodology needs to be developed that addresses the concept and provides some guidance in answering this question. The intent would be to find ways to link, methodologically, the clinical level cost-effectiveness studies to more general policy questions regarding the intensity of intervention in this field and its overall cost to our economy.

A methodological overview has been presented of techniques and approaches that may be useful in ascertaining the cost effectiveness and cost benefit of substance abuse treatment programs. In addition, the type of data that are required to do effective cost analyses is described. Important policy questions have been raised related to identifying the most effective sequence of programs over time for chronic abusers, as well as the optimal allocation of resources that should be devoted to drug abuse. Given the competition that exists in the health and welfare domain for limited resources, those in the substance abuse field will need to develop tools to identify which programs provide optimal treatment in the most cost-efficient manner and what level of programming is appropriate from a cost-beneficial societal view.