Abstract
Background. Treatment fidelity is essential to methodological rigor of clinical trials evaluating behavioral interventions such as Mindfulness Meditation (MM). However, procedures for monitoring and maintenance of treatment fidelity are inconsistently applied, limiting the strength of such research. Objective. To describe the implementation and findings related to fidelity monitoring of the Mindfulness-Based Relapse Prevention for Alcohol Dependence (MBRP-A) intervention in a 26-week randomized controlled trial. Methods. 123 alcohol dependent adults were randomly assigned to MM (MBRP-A and home practice, adjunctive to usual care; ) or control (usual care alone; ). Treatment fidelity assessment strategies recommended by the National Institutes of Health Behavior Change Consortium for study/intervention design, therapist training, intervention delivery, and treatment receipt and enactment were applied. Results. Ten 8-session interventions were delivered. Therapist adherence and competence, assessed using the modified MBRP Adherence and Competence Scale, were high. Among the MM group participants, 46 attended ≥4 sessions; over 90% reported at-home MM practice at 8 weeks and 72% at 26 weeks. They also reported satisfaction with and usefulness of MM for maintaining sobriety. No adverse events were reported. Conclusions. A systematic approach to assessment of treatment fidelity in behavioral clinical trials allows determination of the degree of consistency between intended and actual delivery and receipt of intervention.
1. Introduction
Procedures to monitor and enhance treatment fidelity are the necessary methodological components of clinical trials that help ensure internal and external validity and reliability of behavioral interventions [1, 2]. Systematic implementation of strategies to maintain and optimize intervention fidelity is prerequisite for establishing credible data, drawing conclusions on intervention efficacy or effectiveness, and future replication in other studies or dissemination in clinical practice [2].
Drawing on existing evidence, guidelines, and experience of content experts, the National Institutes of Health (NIH) Behavior Change Consortium’s Treatment Fidelity Workgroup [2] conceptualized fidelity as consisting of several methodological areas: strategies to address treatment integrity (Was the study design and the intervention appropriate for the research question? Was the intervention delivered as intended?); strategies to assess and improve treatment receipt (Were the participants able to understand and perform intervention taught techniques during intervention delivery?); and strategies to evaluate and optimize treatment enactment (Were the participants able to apply the intervention-taught techniques in “real-life”?). Gearing et al. [3] further delineated core ingredients of fidelity that should be assessed as an integral part of behavioral intervention research.
Following the growing public, clinical, and scientific interest in mind-body approaches for general well-being and a variety of health problems, Mindfulness Meditation (MM) programs, often based on the Mindfulness-Based Stress Reduction (MBSR), Mindfulness-Based Cognitive Therapy (MBCT), or Mindfulness-Based Relapse Prevention (MBRP) models [4–6], have been incorporated into the clinical care for substance use disorders (SUDs) [7]. The conceptual framework described in the literature supports the application of MM for relapse prevention in SUDs and suggests that combining MM with traditional, standard-of-care cognitive behavioral therapy (CBT) techniques for SUDs may produce additive benefits [8, 9]. Although promising, research on the efficacy of MM-based interventions in SUDs is inconclusive and limited by relatively small sample sizes and heterogeneity of used methods, including a variety of targeted populations, tested interventions, and strategies for treatment fidelity monitoring [10–15]. The recent Agency for Healthcare Research and Quality (AHRQ) report called for clinical trials of MM-based interventions to rigorously incorporate both monitoring and reporting on treatment fidelity [13].
To address this gap, we conducted a 26-week randomized controlled trial (RCT; ) evaluating efficacy of the MM-based intervention for alcohol relapse prevention and, using existing guidelines [1–3], developed and applied methods to measure and enhance the intervention’s fidelity. This article describes treatment fidelity related methods and findings, which can be relevant to other behavioral intervention trials.
2. Methods
2.1. Study Design
The main study was a 26-week parallel-arm RCT assessing the efficacy of the MBRP for Alcohol Dependence (MBRP-A) intervention, adjunctive to usual care, and compared to usual care alone, for relapse prevention in alcohol dependence. The efficacy findings will be described elsewhere. The trial procedures were approved by the University of Wisconsin Health Sciences Institutional Review Board and registered with https://clinicaltrials.gov/ prior to participant enrollment.
2.2. Participants/Settings
Participants were alcohol dependent adults in early recovery recruited from eight local addiction treatment centers. Eligibility was determined in a two-step process, based on self-report. The initial screen was conducted by phone; those who “passed” this screen were invited for an in-person final screening, during which the extent of alcohol and drug use and related harms were assessed using the Structured Clinical Interview for DSM-IV (SCID) [18]. Eligibility criteria were age ≥ 18 years; English fluency; a SCID-confirmed diagnosis of alcohol dependence in an early remission (defined by lack of heavy drinking: men 5 drinks/day, women 4 drinks/day [19]; and lack of any drinking on ≥3 consecutive days) of 2–14 weeks; completed at least 2 weeks of outpatient treatment (≥2 therapy sessions/week) in one of the collaborating treatment centers; having a home address, phone number, and ability to reliably participate; absence of regular MM practice, preexisting bipolar or delusional disorders, current pregnancy, and a SCID-determined diagnosis of active (past 2 weeks) drug abuse or dependence; and, because stress reduction was the hypothesized mechanism of MBRP-A’s action, an elevated total score (>13 points) on the 10-item Perceived Stress Scale [20, 21].
2.3. Randomization
The study statistician prepared the randomization envelopes (1 : 1), which were consecutively distributed to participants by the study coordinator after the baseline assessment.
2.4. Study Procedures
Eligible and interested individuals completed the informed consent procedures, then baseline data collection, followed by randomization (64 MM; 59 control participants) and scheduling of the MM participants for the MBRP-A course. Control participants were reminded about their eligibility to receive the intervention after completing the follow-up (wait-list controls). Outcome data were collected at baseline, and 8-week (after intervention; F1) and 26-week (F2) follow-up, and included surveys focused on substance use, drinking-related consequences and psychological health, and serum levels of stress-responsive biomarkers (interleukin-6; liver enzymes) assayed from a venous blood sample.
The study team primarily consisted of the study coordinator and the principal investigator (PI) who also functioned as the main study physician. All participants were contacted by the study coordinator prior to scheduled activities as a reminder, to facilitate protocol adherence and assist with transportation, if needed. They were also contacted by phone by the PI for adherence problems and by the study physician to discuss any concerns (e.g., intent to harm; worrisome symptoms; elevated liver enzyme levels) and were sent a letter with liver enzyme results that included a personalized note summarizing any phone conversations about abnormal findings.
2.5. Study Intervention
At baseline, all participants reported engaging in outpatient therapy for alcohol dependence (“usual care”) that typically included motivational enhancement, relapse prevention, and 12-step facilitation strategies [22], but not MM. They were encouraged to continue usual care per recommendations of their regular clinicians. In addition, the MM participants received the MBRP-A intervention.
With permission and assistance from MBRP’s authors, the MBRP-A intervention was adapted from the existing, manualized MBRP program for SUDs [6] to address the needs of alcohol dependent adults. MBRP-A consisted of eight weekly, therapist-led, manual-driven 2-hour group sessions. It provided intensive training in MM, linking MM to CBT-based relapse prevention strategies to create a foundation for acquiring complementing MM and CBT skills for alcohol relapse prevention, as detailed elsewhere [17]. At the first session, participants received a study binder with session-specific handouts; CDs with guided formal meditations (body scan recorded by the study therapist (FL); a set of CDs with meditations by Kabat-Zinn [23]); and a meditation cushion to facilitate home practice. Each MBRP-A session followed the same format: introduction and review of participants’ home practice (experiences, questions, and concerns); practice of 2–4 different techniques separated by a discussion of participant experiences and MM as a means of coping with challenges (e.g., stress, craving) that may contribute to relapse and a review of home practice recommendations for the following week (Table 1). In addition, experimental participants were asked to practice MM at home throughout the 26-week study (formal practice: 30 minutes/day, ≥6 days/week; informal practice, e.g., “mindfulness of daily activities,” “urge surfing:” daily).
2.6. Considerations and Measures Related to Treatment Fidelity Monitoring
Efficacy-related findings, describing the effects of the MBRP-A intervention on alcohol consumption (primary outcome, assessed retrospectively with the Timeline Followback method [24, 25]), and the severity of drinking-related consequences (secondary outcome, assessed with Drinker Inventory of Consequences [26–28]) will be presented elsewhere. The considerations and measures, detailed below, pertain to fidelity monitoring of the MBRP-A intervention.
Following the existing recommendations [2, 3], we grouped treatment fidelity related methods and measures into four categories (Table 2), addressing the core components: () study/intervention design so that it is rooted in a conceptual model or existing clinical practice and enables hypothesis testing; () standardized training of therapists to ensure appropriate implementation of the intervention; () monitoring and enhancement of intervention delivery so that it is implemented as intended; and () monitoring and improving participant understanding and performance of the taught skills (treatment receipt) and their appropriate application in real-life settings (treatment enactment).
2.6.1. Study/Intervention Design
During the study development phase, a theoretical framework behind the application of MM for relapse prevention in SUDs had already been described, suggesting that MM, especially when combined with standard-of-care relapse prevention CBT strategies, can be effective for SUDs [8, 9], leading to the creation of MBRP, which leverages both MM and relapse prevention CBT techniques [6]. We adapted the MBRP program to serve the specific needs of alcohol dependent adults (MBRP-A) and tested the main hypotheses that MBRP-A will reduce the rates of alcohol relapse and the severity of drinking-related consequences. To address these aims, we considered several study designs and settled on a parallel, two equal-arm 6-month RCT comparing MBRP-A plus usual care to usual care alone; therefore, we enrolled only those who reported engagement in a treatment program for alcohol dependence. We elected to add MBRP-A to usual care, rather than assess MBRP-A as a stand-alone intervention, because alcohol dependence can have serious health consequences if left untreated and evidence-based treatments for alcohol dependence exist. This design facilitated cost-efficient data collection and efficacy analysis but led to differences in “treatment dose” (i.e., therapist contact time, group effect) across the two study arms, as only the intervention group received MBRP-A, therefore, limiting our ability to draw conclusions about “true efficacy” of MBRP-A versus nonspecific intervention effects. If shown efficacious in the present RCT, the next study could compare MBRP-A against an “active” comparison condition so that the “treatment dose” is the same across the study arms. With the aim of evaluating the effects of MBRP-A on relapse prevention, the study targeted alcohol dependent individuals in a recovery of 2–14-week duration, a period during which the risk for alcohol withdrawal is reduced while the risk for relapse is increased [29–31].
Therapists needed to have a degree in clinical psychology, social work, or substance abuse counseling; ≥2 years of experience in mental health and/or substance abuse counseling and group therapy facilitation; ≥2 years of a personal MM practice; and experience teaching MM in group settings. Several candidates were interviewed by the PI who then selected one primary therapist (FL) and one main back-up therapist (VGS).
The intervention manual for MBRP-A (available upon request) was adapted from the MBRP’s curriculum [6] and provided a scripted protocol for intervention delivery. The MBRP intervention [6] included MM, with its curriculum patterned after the MBSR [4] and MBCT [5] programs, and relapse prevention CBT strategies [29, 30, 32, 33] for SUDs. At the time of our project start date, no conclusive data were available for the MBRP’s efficacy. With the study focus on alcohol dependence, MBRP’s manual [6] was adapted by the PI (with Alan G. Marlatt (deceased) and other authors’ permission) to the needs of our study population. The PI (AZ), a family medicine and addiction medicine physician and MM practitioner, received training in the MBRP delivery from the MBRP’s authors. The initial MBRP-A manual (Version 1) was pilot-tested in a 16-week uncontrolled clinical trial (), which showed promising results [17]. Afterwards, the manual was revised (Version 2), then retested in an 8-week uncontrolled clinical trial (; unpublished results), and refined further (Version 3) prior to its implementation in the RCT. All revisions were finalized by the PI and informed by the therapist and participant feedback, the PI’s direct observation of the intervention delivery in both pilot trials, and expert input [6, 30, 32–34]; disagreements were resolved by consensus, with Dr. Marlatt, an expert in MM and CBT-based relapse prevention, designated to make decisions in case a consensus was not reached.
An a priori plan for potential implementation set-backs was developed. A plan for when to cancel/reschedule an intervention session was prepared. Two trained back-up therapists (VGS served as the main back-up therapist) were available to replace the primary therapist if needed. A safety protocol was developed, with all staff trained in its implementation by the PI in how to approach participants presenting with worrisome medical or mental health symptoms; the safety protocol binder, containing standardized materials, was available at each intervention session. In addition, to ensure personnel and participant safety and facilitate high-quality intervention delivery, at least two study team members (the therapist and additional research staff) were present at the intervention sessions, which were held in a quiet conference room of a centrally located community hospital with the proximity of emergency and security services. Guided by our pilot study experience [17], the sessions were scheduled in the evening to accommodate a typical work schedule, and light snacks and water were provided to improve adherence. No breaks were scheduled during the sessions; smokers were encouraged to abstain and consider cessation.
2.6.2. Therapist Training
The primary therapist (FL) had 26 years of experience in mental health and substance abuse-related counseling and over 29 years of experience in MM practice, including instruction and a 7-day residential training in the MBSR delivery. The main back-up therapist (VGS) had a similar background and experience. The therapists received a protocol-driven training from the PI prior to delivering the intervention in the RCT. This training included a review of the intervention manual by the therapist, followed by a PI-led one-day, intensive workshop on MM theory, MM application to alcohol relapse prevention, and the review of the manual content and approach to intervention delivery, including role-play practices. The PI was present during the first study intervention (eight sessions) to ensure the primary therapist’s competence and adherence, and provide feedback. The back-up therapist cofacilitated delivery of one intervention (eight sessions) with the primary therapist. The therapists also completed professional residential training in the delivery of MBRP offered by its developers. The primary therapist had also delivered the study interventions in two pilot trials led by the PI. In addition to the training in intervention delivery, both therapists and research staff were trained in the rationale for and implementation of the safety protocol by the PI and the research coordinator during a 2-3-hour in-person meeting.
2.6.3. Intervention Delivery Monitoring
We implemented several strategies to maximize treatment fidelity. One therapist was selected to deliver all intervention sessions, with a back-up therapist available as needed to minimize the “therapist effect.” The PI observed the delivery of the first intervention (8 sessions) and was present at the beginning of each first MBRP-A session to meet with the therapist, introduce herself and the study to the participants, answer questions, encourage adherence and reporting of concerns, and thank participants for their time and effort. She was readily available to the therapist by phone and e-mail throughout the study.
Therapist Adherence and Competence. The therapist adherence (ability to use the strategies outlined in the intervention manual) and competence (the quality of implementation of these strategies), important components of treatment fidelity [35], were formally assessed with the modified MBRP Adherence and Competence (MBRP-AC) scale [36] for the delivered 10 MBRP-A courses. A trained research staff member, present at the intervention session, filled out the scale, rating the therapist adherence and competence after each session with scoring sheets submitted to the PI weekly. In addition, the intervention sessions were audio-recorded; recordings of all sessions from the first, “middle” (5th), and last (10th) interventions and two randomly selected sessions from the remaining courses were audited by a PI-trained research staff (JS) who scored therapist adherence and competence using the modified MBRP-AC scale [36]. The PI audited randomly selected individual sessions; in case of a score discrepancy between the main rater and the PI, the session was audited again and discussed until a consensus was reached. Feedback obtained through audits was communicated back to the therapist by the PI.
The original MBRP-AC scale, developed for treatment integrity monitoring of the MBRP intervention, was adapted and modified by the PI to both align its “checklist” with the content of the MBRP-A intervention and improve scoring efficiency. The audited audio-recorded sessions were scored using the modified MBRP-AC scale [36], which evaluates two session-specific dimensions of therapist performance: competence and adherence. Rating of the therapist adherence to the Key Concepts was accomplished while listening to a recording. Rating of other aspects was conducted after the session had been audited in its entirety and written notes, taken while listening to the recording, had been reviewed. For the scoring of the competence dimensions, the coder assumed a beginning score of 3 and moved to a higher or lower score from there as appropriate. If a portion of the recording was missing, preventing scoring of a given item, that item was left unscored (missing value). To enhance scoring accuracy, once all items were scored, the evaluator reaudited the session to “double-check” any suboptimal ratings or missing values.
Using the modified MBRP-AC scale, therapist adherence was assessed by rating the following (Table 3): (A) key treatment components, itemized in a session-specific checklist (1 = present; 0 = absent); and (B) discussion of the four Key Concepts during the session (awareness of the current experience; acceptance of the current experience; acceptance versus aversion; acceptance and action) rated on a 0 (not explained) to 3 (completely explained) scale. Therapist competence was evaluated by rating the following (Table 3): (A) Therapist Style/Approach in general and in the four mindfulness-related competence areas (inquiry or ability to elicit feedback and respond to verbal and nonverbal feedback; attitude or ability to model and embody the spirit of mindfulness; use of key questions or extent to which they were used to elicit discussion about practices/experiences; use of clarifying questions or extent to which the therapist addressed and clarified ideas or misconceptions about MM), rated on a 1 (absence of the desired style/approach) to 5 (consistent presence of such style/approach) scale. (B) Overall Therapist Performance in terms of the quality of intervention delivery in four areas (the overall quality of the therapy during the session; therapist/researcher ability to work as a team; therapist ability to keep the session focused and on topic; and the overall quality of MM training delivery), rated on a 1 (not satisfactory) to 5 (excellent performance) scale.
The therapist received regular feedback from the PI, stemming from the direct observations, findings from the modified MBRP-AC scoring and participant feedback throughout the study to ensure protocol competence and adherence, and prevent “therapist drift.”
2.6.4. Treatment Receipt and Enactment
Adherence to the intervention protocol was assessed among the MM group participants by their attendance at the intervention sessions (recorded by the research staff), completion of session-specific “homework” worksheets (discussed during a subsequent session), and participant reports (logs) of home MM practice. If a participant missed a session, they received a follow-up call from the study coordinator and offered a brief (approximately 15 minutes) in-person or by-phone meeting with the therapist prior to the next session to “catch up” and encourage continued attendance and home practice. To obtain data on home MM practice, the participants were provided with “calendar” log sheets and asked to log practice minutes daily during the MBRP-A course (with logs collected weekly) and, following the MBRP-A course, to log weekly the average number of days/week and minutes/day of formal practice, and number of days/week of brief, informal MM practice, with logs collected at the follow-up assessments. These data enabled calculation of the total number of minutes of formal and times of informal practice per week. Participants were encouraged to engage in a formal practice 6 days/week, 30 minutes/day, and in informal practice daily. Participant understanding of the taught concepts and techniques (treatment receipt), and skill application (treatment enactment) were assessed through the means of a therapist-led discussion during each session about participant questions, concerns, views, and experiences, including application of the MM skills to relapse prevention and participant engagement in home practice; practicing implementation of the taught skills in hypothetical high-risk situations during the sessions; and completion and review of skill-reinforcing worksheets.
Treatment experience and satisfaction of the MM group participants were assessed at the last MBRP-A session by the Treatment Satisfaction Survey, developed by the research team and successfully implemented in our prior study [17]. The survey contained four questions rating a participant experience on an 11-point Likert scale (0 = “not likely at all/not important,” 10 = “very likely/very important”): () How important this meditation course has been to you?; () How useful this course has been in helping you maintain sobriety?; () How likely are you to continue a formal meditation practice in the future?; and () How likely are you to continue brief mindfulness practices in the future? At the 8-week follow-up, the MM participants also completed three Global Assessment of Treatment questions about their satisfaction with the received MBRP-A intervention (1–7 Likert scale: 1 = “extremely dissatisfied,” 7 = “extremely satisfied”); change in their alcohol problem compared to preenrollment (1–7 Likert scale: 1 = “very much worse,” 7 = “very much improved”); and helpfulness of the intervention for their alcohol problem (1–5 Likert scale: 1 = “very helpful,” 5 = “not helpful, and has made things worse”). In addition, the participants were encouraged to report side effects/adverse events at each contact point.
2.7. Statistical Analysis
Data were double-entered into the secure MySQL relational database and analyzed using the SPSS statistical program (version 23). Descriptive statistics were used to describe data, with results presented as a mean ± standard deviation (SD) or the number (percentage), unless indicated otherwise. The success of randomization was assessed by comparing the groups at baseline with the Mann–Whitney -test for continuous data and Chi square test for categorical data; a two-tailed was considered statistically significant.
3. Results
3.1. Study Sample
During the recruitment period (January 2010–January 2012), 292 individuals were screened. Of those, 36 declined participation, 133 were ineligible, and 123 were enrolled (64 MM, 59 control groups). Participants were, on average, years old, 90.2% Caucasian, and 43.1% female. Approximately 29.3% were unemployed and 44.7% reported an annual income <$20,000. At baseline (12 weeks prior to the quit date), participants reported 59.8 ± 34.8% of drinking days and 50.7 ± 35.3% of heavy drinking days, with 95.9% reporting heavy drinking. There were no significant differences between the groups at baseline on main outcomes, including those related to drinking ( values: 0.406–0.916) and drinking consequences ( for “lifetime,” and for “past 3 months,” total scores). Outcome data were provided by 123 participants at baseline; 107 (53 MM) at F1, and 98 (47 MM) at F2. Ten (7 MM) participants withdrew from the study (91.1% retention rate), with nine reporting lack of time or a scheduling conflict and one withdrawn by the PI due to mental health-related inability to participate in study activities.
3.2. Intervention Delivery Monitoring
3.2.1. Therapist Adherence and Competence
Over the course of the study period, 10 MBRP-A intervention cohorts (eighty sessions altogether) were delivered; 77 of these sessions were delivered by the primary therapist (FL) and 3 by the back-up therapist (VGS). Each intervention cohort included from 5 (the smallest cohort) to 18 (the largest cohort) participants. Among the 59 control group participants, 53 completed their 26-week follow-up and were able to receive the MBRP-A intervention (wait-list controls); 37 of these individuals elected to receive the intervention and participated in the MBRP-A training. The last two MBRP-A cohorts were comprised of the control group participants only.
Audio recordings of the sessions from the first, middle (5th), and last (10th) cohort (a total of 21 sessions, as the recordings of 3 sessions were unavailable due to the equipment malfunction: session 2 of cohort 4; session 8 of cohort 5; session 8 of cohort 10), and audio recordings of two randomly selected sessions from each of the remaining 7 MBRP-A cohorts (a total of 14 sessions) were audited. Therapist adherence and competence during the audited 35 sessions were scored with the modified MBRP-AC scale.
As presented in Table 3, on average, across the 35 audited sessions, the therapist adherence to the Key Treatment Component checklist, which itemized 8–10 key treatment components per session, averaged 90 ± 10%. A 100% adherence was achieved during nineteen audited sessions; 90% adherence during eleven sessions; 80% adherence during three sessions, and 70% adherence during two sessions, with lower adherence related to omitted key treatment components in Session 5 (Coping Cards, Mindful Walking in cohort 1) and Session 7 (Coping Cards, Coping Styles or Mindful Walking in cohorts 1, 5, 6, and 9). With corrective feedback, adherence increased to 90% for Session 5 in the subsequent cohorts and for Session 7 in the final cohort. Adherence to the Discussion of Key Concepts (rated on a 0–3 Likert scale) was scored at points, with the average scoring similar across the four domains (awareness of the current experience: ; acceptance of the current experience: ; acceptance versus aversion: ; and acceptance and action: points). Therapist competence score (1–5 Likert scale) for Style/Approach averaged points across all cohorts (Table 3), with similar scores across the four competence areas (inquiry: ; attitude: ; use of key questions: ; use of clarifying questions: 4.9 ± 0.3 points). The therapist Overall Performance score averaged across all cohorts, likewise with similar scores across the assessed domains (overall quality of the therapy: ; therapist/researcher ability to work as a team: ; therapist ability to keep the session focused on the topic: ; overall quality of MM delivery: points). Of the 35 sessions scored, only 3 sessions (one per cohorts 4, 6, and 7) did not receive maximum therapist competence scores. No trend was identified to associate lower adherence or competence scores with a specific intervention cohort or session.
3.3. Intervention Receipt and Enactment Monitoring
3.3.1. Session Attendance
Among 64 MM group participants, 7 did not attend any intervention sessions (one withdrew from the study; one reported a scheduling conflict; five did not provide a reason) and 57 attended at least one MBRP-A session. In the latter group, 11 attended up to three sessions (with 6 attending only one session); among these participants, all but one stopped attending in the first half of the intervention; and 46 attended four or more sessions (with 22 attending seven or eight sessions).
3.3.2. Home Practice
Sixty-one MM group participants provided reports on home MM practice at F1 (postintervention) and 54 provided reports at F2 (26-week follow-up). At F1, 95.1% of the reporting participants reported a formal ( minutes/week over days/week) and 90.2% an informal ( days/week) practice. At F2, 72.2% of the reporting participants reported a formal ( minutes/week over days/week) and 72.2% an informal ( days/week) practice.
3.3.3. Treatment Satisfaction and Experience
(Table 4) Forty-six MM group participants completed the Treatment Satisfaction Survey at the last intervention session, rating the intervention (0–10 Likert scale) as “important” () and “useful” for maintaining sobriety () and stating they were likely to continue both formal () and informal () MM practice at home. Forty-eight MM group participants completed the Global Assessment of Treatment survey at F1 assessment, reporting the overall satisfaction with the MM intervention (; 1–7 Likert scale, with 7 = “extremely satisfied”), indicating that their “alcohol problem” improved since their study enrollment (; 1–7 Likert scale, with 7 =“very much improved”) and rating the intervention as helpful for their “alcohol problem” (; 1–5 Likert scale, with 1 = “very helpful”). No trend in the treatment satisfaction and experience scores was identified across the intervention cohorts.
3.3.4. Intervention-Related Side Effects/Adverse Events
There were no significant concerns about participant safety during the study, and no serious or unanticipated adverse events were noted among the MM or control group participants. Only one MM participant reported a transient (resolved by 26-week follow-up) worsening of nightmares and anxiety in relation to the formal MM practice. This participant was assessed by the PI who determined the symptoms did not raise concerns for safety. Worrisome symptoms displayed by two participants during the intervention session (they were noted by other session attendees to have a smell of alcohol, without appearing intoxicated) were deemed as unlikely to be intervention-related; following the study safety protocol, these participants were assessed by the PI for disposition.
4. Discussion
Treatment fidelity monitoring is essential to methodological rigor of clinical trials evaluating behavioral therapies, especially complex interventions, such as MBRP-A. However, procedures for monitoring and maintenance of treatment fidelity have been inconsistently applied, limiting the strength of such research. Following the recommendations of the NIH Behavior Change Consortium [1], the present study, a 26-week RCT, systematically implemented procedures for fidelity monitoring of the MBRP-A intervention tested for alcohol relapse prevention. The study intervention was rooted in both conceptual framework and clinical practice and followed a detailed manual, fine-tuned in pilot studies. The protocol-driven approach to therapist selection and training, and the incorporation of the methods to evaluate participant treatment receipt and enactment, increased confidence that the intervention was skillfully delivered and appropriately understood and applied by the participants. The assumption of skillful intervention delivery was corroborated by the findings stemming from the measurement of the therapist competence and adherence to the treatment protocol.
Use of the existing MBRP-AC scale [36], adapted to the needs of this study, enabled a structured approach to the assessment of therapist competence and adherence, integral elements of ensuring treatment integrity [35]. Quantification of therapist adherence and competence allowed us to identify early signs of therapist skill “drift” and, through corrective feedback, to resolve this unintended variation in intervention delivery. The original and the modified MBRP-AC scales could be applied, with adaptation, to other studies of behavioral interventions, especially those based on the MBRP model, to provide a rigorous monitoring of intervention delivery.
Limitations. Several study design elements, which fall within the spectrum of treatment fidelity considerations, could impact our ability to draw conclusions from this study. Though self-reporting on alcohol consumption and MM home practice is the gold standard for this type of study [37, 38], the use of self-reported data can introduce a reporting/recall bias [39], especially when the recall window includes an extended period of time, such as during the final study follow-up. Although the reliance on one primary therapist to deliver the study intervention limits the therapist effect, it also makes it difficult to draw firm conclusions about the generalizability of the intervention’s efficacy, as it does not allow for a separation of the therapist-specific effect from the intervention effect, and the findings will be confounded by the therapist effect in this early-stage RCT.
Future Research. Future research evaluating behavioral or behavior change interventions, including MM-based modalities, should strive to incorporate methods for treatment fidelity monitoring with respect to design, interventionist training, intervention delivery, receipt, and enactment. Although the specifics of a given study/intervention design as well as financial, personnel, and other constraints may determine which of the recommended procedures are appropriate and feasible, the development of treatment fidelity procedures should start during the study development stage.
5. Conclusions
A systematic approach to assessment of treatment fidelity in behavioral clinical trials allows determination of the degree of consistency between intended and actual delivery and receipt of therapy.
Disclosure
The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
Dr. Zgierska’s work was supported by the National Institutes of Health (NIH) National Institute on Alcohol Abuse and Alcoholism (NIAAA) Award K23AA017508. The project was also supported by the Clinical and Translational Science Award (CTSA) program through the NIH National Center for Advancing Translational Sciences (NCATS, Grant UL1TR000427).