Abstract

The present research reports the evaluation of a counseling program for substance abusers in outpatient substitution treatment. Clients were assessed at two time points to evaluate their development on various variables such as psychosocial functioning and substance abuse. Importantly, central outcome variables were assessed via self-reports and ratings by the primary counselor. Methodologically, the challenge was to estimate program impact without a control group as by German law any client in outpatient substitution treatment must be provided with psychosocial counseling. While results suggest that the program positively influences psychosocial functioning, the level of substance abuse did not significantly decrease during the course of the study (N = 101). Furthermore, because program dropout is a major problem in interventions for substance abusers, logistic regression analysis was employed to identify relevant preconditions which predict future program dropout (N = 172). Although several significant predictors were identified, they only provide a modest increase in prediction performance.

1. Introduction

The assessment of the quality and effectiveness of methadone maintenance treatment (MMT) programs for substance abusers is a long-standing topic in research on addiction treatment. While research consistently demonstrates that MMT for opioid abuse is the most effective way of addressing long-term opioid addiction [1], it has also been shown that the effectiveness is boosted if additional psychosocial counseling programs accompanying outpatient substitution treatment are offered (e.g., [2, 3]).

Patients in MMT are typically burdened with a host of problems and lack the resources and coping strategies to even deal with everyday-life situations such as organizing their daily routines. Often they are impaired in their psychosocial functioning, and missing support increases the probability of relapses and dropouts of the MMT program. Thus, counseling programs accompanying MMT typically focus on stabilizing the psychosocial condition of patients and provide support and counseling regarding homelessness, applying for unemployment benefits and other welfare programs.

However, studies assessing the net impact of such psychosocial counseling programs are rare. The reasons for this lack of systematic research are manifold. First, counseling programs are diverse and rather heterogeneous with regard to the services offered which makes an overall assessment of their effectiveness difficult. Furthermore, in Germany where the present evaluation took place, federal law requires patients in outpatient substitution treatment to be provided with a psychosocial counseling program accompanying MMT [4]. That is, patients who are accepted into MMT are mandated to seek additional psychosocial counseling.

The goal of the present research was, thus, to provide an independent impact evaluation of a particular psychosocial counseling program for opioid addicts in outpatient substitution treatment in a large German city. The present research is part of a greater project, some results of which were already reported elsewhere [5]. However, our prior analyses focused on individual goal attainment and did not include a dropout prediction analysis.

2. Method

2.1. Setting

The study took place at the four different sites which offer the outpatient psychosocial counseling program in a large German city. These sites are run by nonprofit organizations, an ecclesial center and the communal health services, and all sites are staffed with social workers experienced in the counseling of clients in outpatient substitution treatment. Approximately 650 clients were enrolled in the psychosocial counseling program at the time of the study.

In general, the counseling program is aimed at clients who receive outpatient substitution treatment for long-term opioid addiction. The duration of prior addiction varies dramatically with some patients having a record of more than ten years of opioid addiction and others not more than a year. After admittance to the MTT program, clients can choose the site at which they seek counseling and are then assigned to a counselor. In essence, the core concept of the program entails frequent counseling sessions in which participants receive support regarding motivation towards abstinence, improvement of physical and mental health, and various other aspects such as help with applying for welfare programs or establishing contact with the clients’ family. The primary areas of counseling encompass establishment of permanent drug abstinence and the stabilization of the clients’ psychosocial well-being as well as physical and mental health.

2.2. Design and Procedure

Due to legal constraints preventing the implementation of a randomized experiment with a control group, a two-wave design on one sample was implemented. In order to counter the threat of unclear validity of self-reports of socially challenged populations such as homeless, mentally ill, or substance abusers ([6], but see [7, 8]) and to enhance statistical power we had each counselor fill in a parallel questionnaire for each client in the study. This combination of self-report and counselor rating allows for the assessment of the degree to which counselors and clients agree in their evaluation of the clients’ progress.

For the initial assessment, the counselors asked all participants to take part in the evaluation who entered the counseling program within a 3-month period and a specified number of participants who currently received counseling. Sampling only participants who just entered the program would have led to a very small N. If the clients agreed to participate they filled out an informed consent form and then the clients and the counselors filled in the wave-one questionnaire. The questionnaires collected sociodemographic data such as age, years of education, marital status, monthly income, duration of opioid addiction, HIV status, and previous enrollment in MMT programs. Furthermore, current drug-taking behavior was assessed for opioids, cocaine, marijuana, sedatives, and alcohol. Approximately 12 months after the first measurement, the second measurement wave commenced and all clients who filled in the wave-one questionnaire and were still in the program were asked to fill in the wave-two questionnaire. Again, the respective counselors filled in the parallel counselor version of the wave-two questionnaire.

3. Results

Because it is beyond the scope of this paper to present the results of all variables assessed in the questionnaires (e.g., the satisfaction with the counseling program, satisfaction with the physician administering MMT, initial motivation to participate in the program, etc.), we will focus on the central outcome variables the program targets such as psychosocial well-being and substance abuse. Last, we aimed at identifying relevant preconditions which are predictive of future program dropout.

3.1. Description of the Sample

The initial sample consisted of 215 participants receiving psychosocial counseling at the first assessment. Complete data for both waves of assessment (i.e., 4 data sets total per participant) were available for 101 participants.

Participants who were not enrolled in the program anymore at the second assessment were treated as dropouts if their counselor could not reach them anymore, if they had been excluded from the program, if they had moved to another town during the study, or if they were in prison without MMT ( ). Participants who had been discharged from the program because they received some alternative treatment or did not need the MMT anymore as well as participants who had died were not treated as dropouts ( ). For the remaining 43 participants, it was unclear why they did not answer at the second assessment. Taken together, this equals a dropout rate of , which is similar to the dropout rate in a 12-month interval found in other German studies (e.g., [9]).

See Table 1 for a detailed description of the initial sample at the first wave of assessment, the final sample across both time points, and dropouts as well the variables which proved to be significant in the dropout prediction analysis.

3.2. Psychosocial Functioning

In order to assess the degree to which the program affects the clients’ psychosocial well-being, we constructed a scale consisting of 7 items which targeted social and psychological aspects of everyday life (e.g., “How burdened do you feel in everyday life?” “Do you have difficulties organizing your daily routines?”). Reliability analyses revealed that the scale was sufficiently consistent for self-reports and counselor ratings (both Cronbach’s ).

In order to test whether psychosocial well-being changed from the first measurement to the second, we conducted a two by two repeated measures analysis of covariance (ANCOVA) with the factors time of measurement (first measurement versus second) and rating (self-rating versus counselor rating) as within-participant factors and the duration of program participation prior to the first measurement as covariate. This covariate was included in the analyses to correct for differences in the amount of treatment received prior to the first measurement. Only those participants for whom data at the initial assessment and at the follow-up assessment were available were included in this analysis ( ).

The analysis revealed a significant main effect of assessment wave, F(1,96) = 4.55, , , = 0.05, indicating that psychosocial well-being was significantly higher at the second assessment than at the first assessment. This empirical effect size of = 0.05 equals a medium-sized effect [10]. Interestingly, the main effect of rating was not significant, , MSE = 0.22, , = 0.01, suggesting that self-ratings and counselor ratings did not differ in general. The two-way interaction was also not significant, , indicating that both participants and counselors similarly perceived an increase in well-being from the first measurement to the second measurement. The absence of the interaction as well as the main effect of rating is interesting as it implies that both counselors and clients agreed in their general assessment of the clients’ psychosocial condition and also perceived the same increase in well-being during the course of the program.

In order to provide further evidence for the causal role of the counseling program in producing this increase in psychosocial well-being, a variable indicative of program utilization was entered in the next step into the analysis of covariance. The logic was here as follows: if the increase in well-being from the first to the second assessment is independent of the services received in the program, then controlling for service utilization should not change the pattern of results. On the other hand, if the program has a causal role in the increase in well-being then controlling for service utilization should render the main effect of time of measurement insignificant because the variance in change should be completely explained by the degree of service utilization. In order to derive a proxy variable for service utilization, we computed the average number of counseling sessions during the study interval as estimated by the counselor. Indeed, including the average frequency of counseling sessions as covariate rendered the factor of time of measurement nonsignificant, , , , = 0.0, indicating that the increase in well-being was completely explained by the frequency of counseling sessions. None of the other factors was significant, all P’s > .05. Nonetheless, it was possible that only those participants who had an initially higher need for counseling also had more counseling sessions and thus had a higher chance of increasing their well-being. Contrary to this argument, the correlation between the initial status in well-being as perceived by the counselor and the average frequency of meetings was small and not significant, r = 0.125, , suggesting that the average number of counseling sessions during the period of the study was independent of the clients’ initial psychosocial well-being.

In sum, these results suggest that participants in the program experienced a substantial increase in psychosocial well-being during the time in the program and this effect can be explained by the frequency of counseling sessions. Furthermore, counselors and clients neither differed significantly in their view on the initial situation nor in their view on the perceived progress.

3.3. Substance Abuse

An important indicator of the success of MMT and psychosocial counseling is the degree to which substance abuse is reduced. Although concomitant substance abuse with MMT is reason to be excluded from the MMT program, substance abuse is not necessarily detected by counselors.

We assessed the average frequency of substance abuse for following drug categories: opioids, alcohol, marijuana, barbiturates, and amphetamines. Comparisons of frequency of substance abuse at the first assessment and the second assessment with the McNamar-test yielded no significant change in any of the four categories: all . Thus, substance abuse did neither decrease nor increase in the observed time period. Although care was taken that participation in the evaluation was anonymous, it is not clear whether participants were actually willing to respond honestly to such sensitive questions and thus these results must be interpreted with caution.

3.4. Dropout Prediction Analysis

Because dropout of MMT programs is a pervasive problem and knowledge of the factors which increase or decrease the probability of dropout can be valuable in preventing program dropout, we sought to identify variables which predict program dropout on the basis of the wave one assessment. In order to assess which initial client characteristics are predictive of dropout of the intervention program, we conducted a logistic regression analysis with dropout status at wave two (still in the program versus dropped out) as binary-dependent variable. All participants, for whom complete data at the initial assessment as well as information whether they had dropped-out by time of the follow-up assessment were available ( ), entered into the analysis. For the predictors, we considered several variables assessed at the initial assessment such as initial motivation to participate in the counseling program, counselors’ expectations for the client’s situation, and duration of prior opioid addiction. See Table 2 for the results of the logistic regression analysis.

Because this analysis was rather exploratory, the variables were entered stepwise with the forward method. Variables that lowered the odds of program dropout were nationality with Germans having lower odds of dropping out (OR = 0.29), counselor’s expectations about the success of program for the client with a positive evaluation yielding lower odds of dropout (OR = 0.38), and time in the program prior to the first assessment with each month lowering the odds by 0.05. Variables that increased the odds of program dropout were psychosocial well-being with lower scores leading to higher odds (OR = 4.25), and surprisingly the degree of leisure activities (OR = 3.12). None of the other variables considered were significant predictors of dropout.

Although the best fitting model significantly increased the prediction performance of dropout as compared with a base line model without any predictors, the variance explained by the model was rather modest (Nagelkerke = .30), suggesting that the variables selected to predict dropout are not very specific. While it is of course possible that other variables might fare better in predicting dropout from the counseling program, it is also possible that the reasons for program dropout are rather heterogeneous. That is, there might not be a “typical” program dropout but rather many different reasons might lead to the final result of dropping out. This result is in line with prior research which also failed in finding relevant predictors for dropout of MMT programs (e.g., [11]).

4. Discussion

The goals of this research were first, to evaluate the impact of a psychosocial counseling program for substance abusers and second, to highlight the utility of simultaneous collection self-ratings and ratings by counselors. Due to the legal constraints in Germany, however, this net-impact could only be estimated indirectly via a two-wave repeated measures design. Accordingly, little can be said about how the clients might have fared, if they had not received the program [12]. Nonetheless, the results of the present evaluation provide preliminary support for the conclusion that the counseling program is effective in stabilizing and improving the psychosocial condition of the clients and helping them achieve important goals towards abstinence. Our analysis controlling for service utilization provides preliminary evidence that the increase in psychosocial well-being is at least in part due to the program as there was a positive relation between the number of services received and the betterment in outcome variables even when controlling for initial differences.

Substance abuse, on the other hand, appeared not to change during the course of the study. We observed no significant change for the four categories of drugs (heroin, cannabis, alcohol, and cocaine). Thus, the important goal of the program to reduce substance abuse could not be reached, at least in the observed time period. It is possible, of course, that changes in substance abuse take place at a much slower rate and that the psychosocial condition and living conditions first need to be improved before clients are able to reduce their drug taking behavior.

An important methodological insight from this study is the advantage of combinations of self-ratings and ratings from counselors to assess the degree to which clients and counselors share a similar view on the clients’ problems and to rule out threats to the validity of the clients’ responses. Indeed, clients and counselors did not differ significantly in their view on the initial status as well as the improvement in the clients’ psychosocial well-being. This result is far from trivial as a variety of factors could lead to diverging views of counselors and clients, threatening the validity of the self-report data. Importantly, collecting data from clients and counselors significantly increases the statistical power to detect changes between pretest and posttest. Because self-ratings and counselor ratings can be considered dependent measures of the same construct, statistical methods for repeated measures such as paired t-test or repeated measures ANOVA can be employed which substantially increase the statistical power. This might be especially important in the area of substance abuse research because often only small samples are available.

Furthermore, dropout of psychosocial counseling programs is a pervasive problem in MMT (e.g., [11]). In order to gain further insight into the preconditions fostering dropout, we considered a variety of variables which were thought to be predictive of future dropout. Although the logistic regression model improved the prediction performance significantly, the practical relevance of this result is only marginal as the sensitivity was only at 32%. However, this sobering result is in line with earlier studies which failed to identify relevant predictors of program dropout [11]. We argue that it might be difficult to identify a prototypical dropout as potentially a variety of individual factors ultimately lead to program dropout.

Acknowledgments

The authors thank the administrators and counselors at the sites for their support in conducting this evaluation. This research was sponsored by the City of Dusseldorf to independently evaluate their program for substance abusers in outpatient substitution treatment. The design of the study was developed by the authors in cooperation with the City of Dusseldorf and the local administrators of the program. The sponsor of this study had no involvement in the analysis and interpretation of the data, the writing of the paper, or the decision to submit the paper for publication.