Abstract
Refugee newcomers resettled in the United States face numerous challenges impacting their mental health and overall well-being. Despite extensive research on clinical interventions, multimodal interventions with peer-based group models in resettlement contexts remain scarce. Adopting a realist evaluation approach, this study aims to conduct a comprehensive implementation evaluation of a complex community-based mental health and psychosocial support (CB-MHPSS) group intervention, examining its mechanisms and processes while considering the interplay among context, implementing agents, and community settings. Qualitative and quantitative data on the implementation process were collected from 11 refugee agencies, involving trained resettlement staffers (n = 32) and refugee community facilitators (n = 31) who implemented the 31 CB-MHPSS psychosocial groups in 2021. The analysis included fidelity reports, process reflections, and follow-up survey responses, utilizing a structured template based on the CB-MHPSS Theory of Change (ToC) and Medical Research Council (MRC) guidance. Results of an in-depth exploration of complex relations between context and implementation processes reveal the vital role that adaptability to socio-ecological circumstances during the global coronavirus pandemic played in achieving successful implementation of psychosocial group interventions. Striking a balance between fidelity and refinement of intervention of the intervention curriculum emerged as crucial factors for aligning facilitation goals with newcomer needs. This approach maintained the integrity and appropriateness of the intervention, requiring implementers to utilize local knowledge of community resources and their understanding of lived experience of forced migrants and their within the community. Furthermore, exploring intervention barriers and facilitators revealed merits aligned with program change theories and areas for adjustment, while cataloged guidelines addressed community needs, showcasing competence beyond standard checkboxes. Strategic dissemination guidance provided by the central office in a supportive and nonintrusive manner facilitated uptake in a participatory and context-specific manner. This study emphasizes the importance of leveraging community partnerships and local knowledge to result in sustained improvements in the mental health and well-being of refugees.
1. Background
Refugee newcomers resettled in the U.S. face a multitude of challenges that could have adverse impacts on their mental health and emotional well-being. This was primarily attributed to preresettlement traumas and stressors, exposure to war and violence and family losses [1], unfavorable living conditions during forced migration [2], and multiple acculturation stressors during resettlement [3]. Consequently, elevated rates of psychological distress and common mental disorders, including posttraumatic stress disorder, depression, anxiety, and somatic symptoms [4, 5], are widespread. According to the World Health Organization (WHO), an estimated prevalence of 22.21% for such conditions exists among conflict-affected populations [6], majority of whom do not access clinical care due to low healthcare capacity. Despite the significant mental health needs, refugees and asylum seekers continue to face significant barriers to accessing proper care and services even after resettlement to high-income countries like the U.S. due to various challenges, including poverty, discrimination, language barriers, and different beliefs and cultural practices that contribute to increased stigma and further impede access to care [7]. Consequently, such factors lead to health care disparities and outcomes that remain suboptimal [8, 9]. The inadequate and untimely provision of mental health care to refugee newcomers has led to the need for effective multimodal psychological interventions.
Empirical evidence has demonstrated the beneficial effects of clinical interventions, such as cognitive-behavioral therapy (CBT), narrative exposure therapy (NET), and eye movement desensitization and reprocessing (EMDR) [10, 11]. While these interventions have shown promise in addressing individual-level mental health needs, a heavy focus on trauma-related symptoms (e.g., post-traumatic stress disorder) has left a paucity of research examining the efficacy of multimodal psychosocial interventions, particularly those conducted in naturalistic, uncontrolled settings that reflect the complexities of healing processes [7, 11]. To enhance emotional coping, increase access to mental health care, and support community healing responses, implementing multimodal interventions in community settings, particularly those led by refugee peers and delivered in group formats, is considered paramount [12]. Peer-led group-based support has been shown as effective in reducing social isolation and supporting community healing responses [13, 14]. The development of mental health care competencies through group support enhances social integration, reduce feelings of loneliness, and broaden coping strategies among refugees [15]. Community-based group intervention models are promoted worldwide, gaining recognition for their adaptability and scalability in various settings, particularly when supported by safety measures and supervision frameworks [16]. To enhance mental health outcomes and well-being for refugees during resettlement, it is crucial to evaluate multimodal interventions that incorporate peer-led group support in real-world settings.
1.1. Implementation Evaluation of Complex Interventions
Community-led group interventions are a mechanism to embody the Community-Based Mental Health and Psychosocial Support (CB-MHPSS) approach set forth by UNICEF [17]. CB-MHPSS builds upon the established framework of the Inter-Agency Standing Committee’s (IASC) global Mental Health and Psychosocial Support (MHPSS) programming [18, 19]. The framework aims to support conflict/disaster-affected forcibly displaced persons in humanitarian settings, helping afflicted individuals, families, and communities achieve safety and stability and improve their mental health and psychosocial well-being [18, 20]. The multilayered approach set forth by IASC is also applicable in resettled contexts [21, 22]. This approach posits that healing occurs with active involvement and leadership of forcibly displaced persons in defining and implementing culturally relevant psychosocial intervention processes.
Mental well-being is conceptualized as inherently relational individuals realizing their own potential, who can cope with the normal stress of daily life, can function, and contribute to their community; these outcomes are in direct relationship to factors in an individual’s socio-ecological environment [17]. The intervention design developed by the authors advanced that psychosocial support groups utilizing an authored curriculum (e.g., [23] would improve mental wellness and psychosocial well-being of refugees and equip lay facilitators with skills to address a variety of mental health needs of the refugee community, simultaneously, influencing broader, sustainable support structures at the community level. By acknowledging community needs and cultural contexts, this approach empowers community stakeholders to develop interventions that are culturally and contextually appropriate, thereby enhancing their capacity in designing effective solutions. Community-driven approaches in mental health prevention and promotion demonstrate enabling factors that potentiate social support, empowerment processes, and healing among refugee communities [24]. Involving community stakeholders as active agents in intervention implementation process can also promote non-pre-registered or unexpected outcomes critical to improving social determinants of mental health, such as ownership, sustainability, and cultural appropriateness of the intervention, which are crucial to addressing social determinants of mental health [25, 26].
The interplay of various factors, including structured organizational support, informal self-help, community needs, and socioeconomic and political settings, transforms a community-based psychological intervention into a complex, multilevel effort for community awareness and care systems. Therefore, gaining an in-depth understanding of intervention implementation complexities in real-world contexts is critical to better inform practice and service delivery in diverse settings, acknowledging and considering meaningful changes and impacts that affect all community stakeholders involved in the process. In fact, the success of a psychosocial intervention is contingent not only on its effectiveness but also on its ability to reach those in need, integrate into existing services, and maintain sustainability [27]. Understanding the acceptability and feasibility of an intervention is also imperative for its continued delivery within a particular setting and for future efforts to scale up sustainably in other community settings [28].
A range of factors contribute to the complexity of HIAS’s CB-MHPSS psychosocial intervention, such as the multifaceted components of the MHPSS curriculum, composition of intervention groups and community settings, flexibility of program delivery, and the expertise and skills required by bicultural facilitators implementing the intervention. Combined, these characterize it as a complex intervention [29]. These contextual factors intensify the intricacy of the intervention, requiring consideration of their influence to ensure its efficacy and relevance [30]. The complexity of the CB-MHPSS intervention in refugee resettlement extends beyond these factors, encompassing the interactions between the intervention and the broader context of resettlement services, including community and organizational settings, as well as socioeconomic and political circumstances.
To assess such complex implementation processes, we adopted the Medical Research Council (MRC) guidance [29, 31], which allows for integrating intervention processes and mechanisms into comprehensive evaluation approaches [31] and recognizing the influential role of context in shaping implementation and causal processes [32]. Utilizing MRC’s framework, we conducted an appraisal of HIAS’s psychosocial program’s Theory of Change (ToC), paying attention to such key implementation elements as program theory, stakeholders, uncertainties, and refinement, while also taking into account economic considerations. To elucidate the mechanisms by which an intervention brings about change, this study employs the ToC framework furnishing insight into how and why an intervention is effective by providing the theoretical underpinnings of the intervention [33, 34]. This facilitates a more comprehensive examination of the interplay between the intervention and context through numerous causal pathways, levels of intervention, and feedback loops that accurately reflect the manner in which complex interventions yield their impact. The ToC approach also helps elaborate on disparities between anticipated and observed outcomes, identifies how context affects outcomes, and provides insights to improve implementation.
1.2. Current Study
Figure 1 displays the CB-MHPSS ToC that presents relevant activities, outputs, and outcomes as guiding principles of the program implementation. The ToC for CB-MHPSS psychosocial program focuses on three key objectives and components: (1) participant wellness, (2) provider capacity, and (3) community reach. Component one aims to improve participants’ emotional wellbeing and daily functioning, by mainstreaming awareness of MHPSS clinical and nonclinical resources through a 9-session peer lead group intervention. The theory posits that newcomers and refugee cofacilitators will exchange culturally and linguistically appropriate coping resources and resilience safeguards with each other, accounting for local sites’ unique needs and adapting the curriculum for their purpose and population based on numerous variables. The CB-MHPSS group interventions have demonstrated effectiveness in improving emotional well-being, as assessed by the Short Warwick-Edinburgh Mental Well-being Scale, and in enhancing competencies for mental health and psychosocial services, including skills and confidence in navigating MHPSS resources and promoting positive outcomes (Authors, under review).
The second component concentrates on strengthening provider capacity, by assessing improvement and capacity of affiliates alongside community leaders to identify and refer at-risk clients for additional mental health support within the local mental health landscape throughout and after the intervention cycle. This feature involves imparting knowledge through structured capacity-building training and delivery of formal and informal technical assistance provided by the headquarter office with field experts in refugee mental health. Trained affiliates and local staffers also support community facilitators with group programming and build capacity for ongoing identification and referrals of clients to support resources.
Thirdly, the last areas of intervention focus on participant “reach” by distributing psychoeducation about mental health and peer-to-peer psychosocial support, including emotional coping techniques to facilitate resilience, and engagement with community leaders as healing champions with enhanced knowledge, skills, and mutual support. These activities are tailored to the unique characteristics of each local site, considering factors such as cultural context, language, and specific challenges faced by the population. By adapting the curriculum to local needs, the program aims to maximize its effectiveness and promote sustainable outcomes within each community.
In sum, the overarching aim of this study is to conduct a comprehensive implementation evaluation of the complex CB-MHPSS psychosocial program, exploring its mechanisms and processes, taking into account the interplays among context/setting, implementing agents, and participants/community. The CB-MHPSS psychosocial program ToC in conjunction with MRC’s guidance is used to present the successes and gaps in anticipated and unexpected output/outcome, examining the barriers and facilitators of implementation.
2. Methods
2.1. CB-MHPSS Psychosocial Group Implementation
As depicted in Figure 2, the psychosocial program comprises multiple phases of implementation. These include capacity building training, curriculum development and modification, pilot implementation and scaling up, and an “expansion” phase which responded to supporting Afghan evacuees to the United States. The program utilized a HIAS-authored MHPSS curriculum, developed through collaboration between global MHPSS experts and the HIAS headquarter office, in conjunction with three volunteer affiliates who conducted a pilot program with seven groups consisting of 58 refugee/asylum seeker participants in 2020. Technical assistance was provided to affiliates by the HIAS team and authors, who imparted a training package developed by HIAS that covered topics related to group teaching and education skills, basic MHPSS principles, and skill-building relevant to curriculum teaching. In addition, biweekly sessions were held among affiliates to enhance mutual learning and provide bidirectional, interactive support. The training and mentorship resulted in the compilation of a replication guidance, which included teaching tools for safety and risk, deescalation, mapping of local mental health referral pathways, and other capacity-building tools related to mental health programming.
The delivery of the intervention was planned and executed to uphold effectiveness and fidelity throughout the process. A diverse coalition of peer facilitators, bicultural staff, and specialized volunteers collaboratively cofacilitated the program, each bringing a unique set of skills and perspectives. These facilitators were comprehensively trained over a series of at least four sessions, ensuring they were well equipped with the necessary knowledge and expertise to proficiently deliver the program. In response to the varying circumstances presented by the external environment, the program was primarily conducted remotely, demonstrating necessary flexibility and accessibility in its mode of delivery. This adaptability was crucial in navigating the complexities and uncertainties brought about by the prevailing pandemic conditions. The curriculum, initially informed by the Pathways to Wellness (P2W) program, underwent an adaptation process during the initial pilot phase. This was a collaborative endeavor involving implementers, peer reviewers, and consultants, driven by the evolving needs and challenges faced by affiliates during the pandemic. Through iterative cycles of revision, review, and refinement, the curriculum was fine-tuned to meet the real-time needs and circumstances of the communities being served, ensuring its continued relevance and applicability amidst the dynamic landscape of the ongoing pandemic.
Three HIAS affiliates mentored a cohort of 11 affiliates through a peer mentorship model, providing individualized coaching and training support to other implementers in the community and other affiliates, including staffers in other affiliates and community volunteers. In 2021, three pilot sites transformed into mentor sites, providing 22 hours of individualized coaching and training support to all 11 implementing sites, followed by 6 hours of expert training from HIAS. Using a peer-mentoring and interactive skill-building approach, mentee sites were able to familiarize themselves with the CB-MHPSS curriculum and core MHPSS competencies and enhance psychosocial skills for intervention facilitation, adoption, and contextualization. The authors converted the interaffiliate training into a written Training Manual that cataloged training curricula for community facilitators, most of whom were refugee community leaders themselves. The local/affiliate-level mentoring process involved nonuniformed training, depending on their needs and capacities for further application and adaptation. A total of 32 trained staff members from 11 affiliate agencies attended training and supported 31 other field facilitators to implement CB-MHPSS groups between February and May of 2021. The intervention was implemented mostly online, with the exception of one affiliate site in a shelter setting, all during the height of the coronavirus pandemic.
2.2. CB-MHPSS Curriculum
The HIAS CB-MHPSS intervention curriculum was developed to provide refugee communities with tools to understand and address their psychosocial wellbeing and mental health needs, while building social networks to support one another. The curriculum is comprehensive, integrating psychoeducation, mental health awareness, stigma reduction activities, family and community resilience building, and coping strategies to address cultural adjustment, with a focus on the collective trauma of living through a global pandemic. In fact, the curriculum was modified after an initial pilot with three sites during 2020 at the beginning of the pandemic. Program designers gathered field notes and experiences from implementing sites to informing content changes. A refugee community member provided a peer-review of the adapted curriculum prior to its release among the 11 implementing peer trainee sites. The curriculum consists of 9 sessions, each lasting an average of 75–80 minutes, and is primarily delivered virtually. It is designed to help forcibly displaced persons cope with the common reactions to adversity and trauma, using principles of trauma interventions [35]. An outline and session structure of the curriculum is provided in Table 1. The curriculum is intended to be delivered by community facilitators who possess lived experience relevant to the community being served. Program designers envisioned a flexible and adaptable structure for the curriculum that would allow for modifications within the specific framework of the themes and scope of each weekly topical area, whereby the intervention empowers refugee communities to build self-help resources and community resilience in the face of adversity.
2.3. Data Collection
The implementation of the group curriculum was executed in 11 distinct sites among diverse immigrant groups from various forced migrant backgrounds. The program was designed to cater to the mental health needs of refugees, asylum seekers, and Special Immigrant Visa holders, with some groups composed of community members who had been resettled for up to five years. The implementation context was challenging due to fluctuations in refugee arrivals and changes in the promotion modality. For instance, some sites had to switch from in-person to remote modalities due to the COVID-19 pandemic. Community mobilization was also challenged, especially in communities that were not used to seeking mental health support. This implementation context was characterized by global, national, and local factors that impacted the delivery of the program. All but one of the affiliate sites implemented multiple intervention groups, with a range of 2 to 5 groups (M = 2.82, SD = 1.45). These groups were conducted either simultaneously or consecutively and were divided by gender or age group, creating a culturally sensitive and secure environment for the participants. In total, 235 refugees and asylum seekers were enrolled in 31 groups across 11 sites, with an average of 7.6 participants per group (93 male, 142 female). The program was staffed with a variable number of trained personnel, ranging from 1 to 10 individuals (M = 2.82, SD = 2.15), while on average, 2.6 community cofacilitators (SD = 1.94), many of whom were refugees themselves, were trained to implement the CB-MHPSS groups.
To ensure fidelity and integrity of the program implementation, weekly feedback checklists were utilized as a primary tool for data collection and analysis. Each implementer, including any group facilitator, completed this checklist at the conclusion of every weekly session. The checklists were crucial in monitoring the completion, alteration, and adherence to various session topics, capturing specific modifications made and the rationale behind them. This approach facilitated a consistent and systematic collection of data, allowing for a real-time analysis and enhancement of the program’s delivery and effectiveness. Additional contextual information and fidelity checklist details are available in Table 2.
2.4. Data Analysis
Since this study utilized data originally collected by HIAS for program evaluation purposes, a secondary dataset was employed by our research team, which was anonymized and deidentified to adhere to rigorous ethical standards. The study protocol underwent a thorough review process by the research institutions of the first and second authors and was granted an exemption, ensuring the objective and ethical conduct of the research. A structured template was utilized for organizing qualitative data, including weekly fidelity reports, biweekly process reflection and final reports, and open-ended follow-up survey responses. The CB-MHPSS ToC was the basis of the template, with specific attention to the process evaluation elements outlined by the MRC [31]. Emergent themes were identified and organized into each ToC assumption through a content analysis embedded to a framework approach [36]. The MRC framework guided the elaboration of complex relations between context and implementation process/mechanisms [29]. Specifically, the three implementation components that are often implicated but not explicitly articulated or elaborated on in the ToC were scrutinized, namely, interactions between the intervention and context (including community/organizational settings, accounting for the pandemic, economic recession, and racial trauma), flexibility in delivery, adaptation of curriculum, and capacity and capabilities of implementing and receiving agents. Two members of the research team independently read through the data to identify codes related to each ToC component, which were collated and organized into each domain and theme. Discrepancies were resolved through discussion until consensus was reached, which was then scrutinized by the rest of the research team. Descriptive statistics were conducted to analyze quantitative fidelity data, preintervention and postintervention, and follow-up survey responses. Frequencies of responses for each open-ended question were computed to understand the distribution of responses across intervention groups and affiliate sites, allowing for the identification of any patterns or trends in the data. Analysis delineated percentages of sessions that were conducted live, remotely, and in a hybrid format, providing a comprehensive understanding of the implementation landscape and its subsequent impact. This approach ensured that the program was broad-reaching and adaptable, capable of navigating the complexities and challenges inherent in diverse delivery modes and settings. All statistical analyses were performed using IBM SPSS Statistics (WIN version 27).
3. Results
3.1. Feasibility and Acceptability of CB-MHPSS Group Intervention
Participants, including both refugee/asylee participants and implementers (i.e., affiliate staff and community cofacilitators), provided overwhelmingly positive feedback on the CB-MHPSS intervention. They appreciated the curriculum’s thoroughness and comprehensiveness, as well as its clear structure and point of reference for weekly sessions. The curriculum covered a wide range of topics that aimed to encourage conversations about emotional health, addressing areas that community members may not have previously felt comfortable discussing. All affiliates reported positive impacts in various domains, such as enhanced relationships with the refugee community, earned trust, expanded partnerships and care resources, facilitated referral processes, and increased awareness and openness to emotional and mental health care. Both qualitative reports by implementers and feedback from participants strongly supported the positive impacts of the intervention for refugee participants, with a majority reporting improved emotional well-being (68.6%), confidence in understanding more about mental health care (55.3%), and increased comfort in supporting others with mutual aid or peer support (45.2%). A significant number of participants also appraised the CB-MHPSS curriculum as valuable and recommended it for other newcomers. The transformative experiences of the participants were evident across all sites, although the implementation of the intervention groups varied to some degree across the 31 groups. All 11 affiliates documented remarkable increases in newcomers’ comfort levels to discuss and address emotional and psychosocial challenges, including coping during the pandemic. Participants perceived groups as safe spaces for newcomers to share personal stories, express emotions, and provide support to one another.
3.2. Implementation Barriers and Facilitators
The findings reveal several implementation barriers and facilitators that play a critical role in ensuring or obstructing the effective implementation of the CB-MHPSS intervention at the group, agency, and community levels (refer to Table 3 for a comprehensive list of content analysis with example quotes).
One key enabling factor of successful implementation was the extensive training and peer mentorship meetings that reached not only service providers but also community leaders and cofacilitators. The training of facilitators, including agency staffers and community leaders, was deemed essential not only in equipping them with the necessary knowledge, skills, and competencies to deliver the intervention effectively but also in encouraging and embracing cultural insights and community relationships to make the intervention culturally appropriate and accessible to the target population. The facilitators of the interventions appreciated biweekly debriefing sessions and the opportunity to learn from other affiliates. The program was also regarded as valuable in expanding their “toolbox” of different ways to promote community mental health. Furthermore, the frequent supportive supervision and mentorship calls created a knowledge repository and a source of knowledge exchange that facilitated confidence, growth, and practical learning among implementers.
The affiliate peer mentorship was very helpful. Each agency of our group was at different steps of the process of implementing the groups, so we were able to ideas off each other, provide advice, and feedback on new ideas. Our groups were successful due to the resource of the affiliate peer mentorship (Site A).
The institutional structure and existing capacity of resettlement sites were also found to be significant facilitators. Each affiliate agency served as central “hubs of services,” making the community-based MHPSS model well suited for uptake and adoption into the local context. The localization of the implementing agency within a milieu of refugee social services facilitated the program’s implementation, allowing intervention integration into existing services and structures. However, site capacity, resources, and the composition of forced migrant and community facilitator skill and background vary significantly. It is important to note that even within these existing structures and programs, psychosocial care or mental health service components are often lacking or nonexistent (i.e., 36.4% of in-house clinical referrals), and therefore, community resources for culturally relevant care were critical, especially for recruitment and retention of both participants and cofacilitators, as well as the implementation sustainability, yet they differ widely across sites. Some sites further built capacity through enhanced partnerships and networking, hiring a therapist who speaks the same language as the refugee target group to respond to in-house mental health referrals. In another location, the implementing agency noted an increase in specialty referrals to partners such as a Survivors of Torture program.
Another significant enabler was the inherent flexibility of the psychosocial programming, which facilitated cultural and contextual adaptation, alongside technical assistance from HIAS and resident expertise through peer mentorship support from affiliate agencies who had previously implemented the group project facilitated successful adaptation.
We ended up skipping a couple of the set curriculum sessions to just have a chance for the women to have a relaxing space to get to know one another and to share with us videos and pictures from their home countries/cultures. My co-facilitator and I realized that this was such a unique opportunity for cross-cultural learning and exposure to different forms of thinking, food, and art. We also discussed cultural understandings and norms around mental health and wellness and health. (Site J)
Despite successful completion of the CB-MHPSS intervention in each site, its implementation proved to be a complex and challenging process, especially during the early phase of the COVID pandemic. The most significant barriers encountered included recruitment and retention, the need for tailoring both the curriculum and modality, and challenges in organizational or community capacity and competency. Some participants stated that they had to drop the program due to their inability to dedicate enough time, suggesting that time constraints were a major obstacle that resulted from competing resettlement priorities. Similarly, others faced challenges related to the program’s digital platform and content-heavy curriculum, which could be customized for open dialogues and further localization in order to integrate new referral tools into their other programs for newly resettled refugees. Assessing the program’s efficacy posed difficulties, especially concerning the preintervention and postintervention surveys. Some refugee participants found surveys confusing, and those identifying as asylum-seekers expressed concern about potential repercussions for not completing them. The stakeholders’ divergent priorities further complicated the survey process, resulting in incongruity in response rates particularly among asylum seekers. A facilitator stated,
The group members felt the pre and post surveys were monotone and repetitive. Participant did not feel that their answers varied much from one session to another. Participants did not feel the need complete each pre and post surveys in a timely manner. (Site K)
3.2.1. Intersecting Barriers and Facilitators
Some factors are deeply intertwined and have the potential to either facilitate or impede progress, both within a single context and across different settings. The diverse composition of groups presented challenges in some locales, despite its value of cross-cultural relation-building and knowledge exchange. Eight localities (72%) including 21 groups (67.7%) involved participants from different cultural and linguistic backgrounds, and although sharing the same collective experience of living during the novel coronavirus, finding additional common ground among them was often challenging. Such diversity of participants made the curriculum’s relevancy different. For example, a site facilitator said, “Some of the concepts outlined did not align with cultural values or perceptions of mental health. Notably, one of our groups elected to skip the session on healthy living because they felt the curriculum did not take into full account the relationship between food access, nutrition, and culture (Site D).” However, other affiliates noted “our groups were very diverse and composed of women from completely different countries, continents, religions, etc. Outside of this group, these women had never met each other. [……] My cofacilitator and I realized that this was such a unique opportunity for cross-cultural learning and exposure to different forms of thinking, food, and art. We also discussed cultural understandings and norms around mental health and wellness and health.” (Site J)
Talking about trauma and stress presented a significant barrier to complying with the curriculum in certain localities, given the nature of psychoeducation and the high stigma around mental health. Many participants found it challenging to share their experiences, making it difficult for facilitators to provide the necessary support. However, some localities had the opposite experience. For example, one affiliate stated, “The biggest lesson learned was how beneficial it is for group members to process trauma with others who have similar experiences to their own” (Site K). Although the vast majority continued to adhere to safe distancing protocols, the rapidly changing nature of the novel coronavirus made it challenging to keep up with new information about COVID-19. To address this, weekly check-ins were provided to discuss recent COVID-19 related measures, which was an encouraging factor. The alignment of intervention focused with the evolving challenges posed by the uncertain pandemic has been acknowledged by participants, as exemplified by one participant’s statement: “this group has helped me learn a multitude of strategies to effectively navigate stress and maintain a positive outlook in challenging circumstances. I express a desire to participate in similar groups in the future.” This response resonates with underlying assumptions surrounding the ToC, in which effectiveness of refining interventions that address the real-world context of emerging distress, demonstrating the potential for ongoing adaptation to support individuals amidst evolving circumstances. Finally, varied levels of competency and capacity among agencies played a role in a unified and global approach to CB-MHPSS psychosocial groups. Some agencies requested more extensive training for cofacilitators, recognizing the need to address potential biases that may influence how they run sessions. In contrast, other agencies with greater internal resources and capacity found the peer mentorship program to be of limited value, stating “While the peer mentorships did provide some help, we only occasionally came away from peer mentorship meetings with new or valuable information to carry forward in the program” (Site B).
3.3. Fine Balance: Fidelity and Refinement
The CB-MHPSS curriculum was well followed: 79% of respondents fully completed the curriculum activities as authored, while 2.5% partially completed them. Interestingly, 11.3% completed the provided activity but added their own modification, tool, or unique change. 10.4% chose not to complete the activity at all. On average, each session took 79 minutes, with the shortest session lasting 61 minutes (Session 8) and the longest lasting 100 minutes (Session 4). In the reports where fidelity to the curriculum was not maintained during the sessions, 44 filled checklists (68.8%) showed “Altered” sessions and 8 (12.5%) were “Rejected.” Among the reported design alterations, 12 (28.6%) were reported as “Change of content, context, or process,” while the majority (n = 30, 71.4%) were “Additions” to the current curriculum. A content analysis of open-ended responses on fidelity checklists yielded four themes of challenges in fidelity: (1) Length of Time (n = 17), where all reported that the curriculum was “Too Long”; (2) Content (n = 24), where 9 (37.5%) reported “Unclear” content, another 9 reported the curriculum as challenging to implement, 6 reported content unsuitable for the population/culture, and 1 reported interpretation challenges; (3) Context (setting and population), where 5 (31.3%) respondents reported challenges with the “Setting,” and 11 (6.5%) reported challenges with the “Population/community”; and (4) other operational obstacles, where the majority expressed difficulties in attendance and attribution (n = 7, 58.3%), more guidance/resources needed (n = 4, 33.3%), and one reported evaluation challenges (8.33%).
3.3.1. Refinement and Adaptation
Given the multigovernance of program operations, it is natural for multiple sites to practice and allow local adaptation and refinement, rather than being a symptom of ineffective management, emergent refinement and changes are considered appropriate, necessary, and even desirable in many cases, including some logistical challenges, community reach, participant requests, and acceptability of curriculum content. Different priorities in objectives emerge through negotiation and conversations during sessions, as well as during debriefings and facilitator meetings, as one facilitator mentioned,
We ended up skipping a couple of the set curriculum sessions to just have a chance for the women to have a relaxing space to get to know one another and to share with us videos and pictures from their home countries/cultures (Site J).
Some modifications to the MHPSSS curriculum were content-based resulting from each diverse population they served, such as supporting asylum-seekers and shelter inhabitants. Different group membership and composition also required specific enhancements. A site that served asylum-seekers or those seeking political asylum modified the curriculum to provide education and information for asylum-seekers who are typically excluded from health insurance and other benefits. Participants reported a transformational experience similar to empowerment processes, with many expressing gratitude for having a dedicated space to process emotions of being stuck in legal limbo. An affiliate recounted,
One of the biggest impacts these groups had was giving people the opportunity to release their stress each week, process their emotions, and engage with different members of the community. Clients stated they felt supported, laughed, played games, shared their stories and were able to work on their wellness by attending these groups (Site F).
The pandemic situation also affected the intervention modality, obstructed or facilitated by the level of digital literacy and acceptability of online platforms by implementers and participants. “I think that we learned a lot of lessons about creating space for connection and trust, especially over Zoom that does not happen naturally” (Site J). Flexibility in scheduling was a key in determining the best method of meeting to maximize participation and attendance. Some groups had to meet in person on Sundays, while others used WhatsApp to host weekend meetings, for example. Extra accommodations as well as attentive support helped overcome the digital literacy issue, as a staffer stated,
We were able to utilize community and family members to assist clients with getting into the sessions. Client comfort with their devices also grew over time. For clients where this was not an option, we found that calling clients directly and then calling into the Zoom meeting with a three-way call allowed clients with limited digital literacy to participate throughout the program (Site B).
As one facilitator noted, “It was really important to use the structure of the curriculum and also create time for unstructured conversation and culture sharing” (Site J). In creating a space for healing, the facilitator stressed the need for additional efforts, including investing more time and locally adapting the curriculum. Several facilitators noted that building community and trust before delving into more sensitive topics that required vulnerability and openness took longer than anticipated. Participants also reported how validating their feelings and experiences was through open conversation and that they gained insight from listening to the experiences of others. The facilitators acknowledged the significance of flexibility in modality and local contextualization of the curriculum. For example, depending on the level of understanding and digestion of the curriculum content, facilitators created and offered extra content to ensure participants’ learning and understanding. A staffer said,
I needed to establish a reason why we are in the Support Group. Then, I brought in more of my own exercises and ideas, still while following the core curriculum themes and discussions each week. When you get people to talk about who they are, it gives them an inner purpose, a connection to their old culture and new culture. When you talk about who you are, it can be a way to channel your stress. [……] When I put them face to face with the stress they had before and the stress they had now, they realized one is better than the other. It creates a space for acceptance, compromise, and resilience (Site F).
In sum, a fine balance between fidelity and refinement is important in order to ensure that the program remains faithful to its original goals and evidence-based principles while also being responsive to the needs and preferences of the target population amidst the changing contexts in which it is implemented. Balancing cultural adaptation with intervention fidelity turned out to be a critical challenge in CB-MHPSS programming, and certain factors, both challenges or uncertainties, drove refinement and modifications of the MHPSSS curriculum and implementation, such as content, modality, reach and inclusion, and monitoring and evaluation.
4. Discussion
This study sheds light on the complex processes involved in implementing CB-MHPSS interventions, revealing multifaceted factors that both facilitate and challenge successful implementation. As previous research [37, 38] has confirmed, cultural relevance of interventions and importance to participants were key factors in facilitating its success. Achieving this often requires additional adaptation and localization, such as modifications in curriculum content, modality, and delivery at the local level, to match facilitation goals with participant needs. In addition, our study brings forth the novelty of flexible, implementer-driven models in CB-MHPSS interventions. This approach allows for a more nuanced and adaptive response to the diverse needs of refugee and asylee populations, filling a gap in the U.S. context where there are limited low-intensity, adaptable mental health intervention models [39, 40]. The skill development in facilitators proved crucial for ensuring a comprehensive understanding and response to these adaptations. This underscores the importance of training modules that emphasize cultural sensitivity, trauma-informed care, and curriculum customization, allowing facilitators to cater to the diverse needs of the refugee/asylee populations they serve. Safe spaces are integral, creating an environment conducive to open dialogue, shared cultural experiences, and a sense of belonging, which facilitates open expression of vulnerabilities. Moreover, digital integration has surfaced as an essential facet, with some participants finding value in platforms such as Zoom and WhatsApp. In fact, the utilization of platforms such as Zoom and WhatsApp was crucial during the COVID-19 pandemic, maintaining program continuity despite physical distancing. This experience echoes emerging literature highlighting tele-health’s transformative role in mental health and psychosocial support (MHPSS) services [41]. Technological advancements have dismantled barriers, such as digital literacy and geographical constraints, enhancing accessibility, inclusivity, and confidentiality in MHPSS services, and fostering resilient, community-engaged services responsive to diverse needs, including those of refugees and asylees. However, the digital approach also revealed challenges, such as reduced social connection due to lack of physical proximity and potential risks to confidentiality, emphasizing the need for a balanced approach in future implementations [42, 43].
The flexibility of the psychosocial intervention for multisite operations that can be scaled presented an additional layer of complexity, yet dissemination and peer learning supported adaptation and refinement making the intervention most effective and suitable for local needs. Community engagement, as evidenced by transformative experiences, underscores the importance of trust-building and confronting existing mental health stigmas. Collaborative efforts with community leaders and cofacilitators, through avenues like community consultation sessions, can ensure culturally relevant interventions. Furthermore, it is essential to address barriers, including time constraints and challenges related to digital literacy, which require strategic integration into any intervention.
Implementing psychosocial support groups also required balancing fidelity to the original curriculum with cultural adaptation to ensure the intervention’s integrity while enhancing its appropriateness for the target population. However, achieving this balance required a deep understanding of the community’s unique needs and configurations of community resources, as well as the implementer’s advocacy for the community and competence beyond required checkboxes. This highlights the importance of not only adapting the intervention to the local context but also ensuring that the components of the intervention are integrated and function together to produce desired outcomes. The implementation of CB-MHPSS groups requires an ongoing assessment of the relationships between the components and the system to identify areas where adaptation and refinement may be necessary [44]. Ultimately, the success of CB-MHPSS groups hinges on a deep understanding of the complex relationships between the parts and the whole and the ability to navigate these relationships effectively to achieve the intended outcomes (ex. realist emergence theory, [45]). Localization policies that take into account the diverse capacities and resources of resettlement sites can ensure more effective and tailored interventions. A one-size-fits-all approach may not address the complex needs of diverse refugee and asylee populations.
Another prerequisite for implementation of a complex intervention through both fidelity and localization is dependent on skill development and community capacity. They include implementer training and in-kind agency support, community engagement and partnership, and trusted relationships with culturally responsive caregivers and services. Policymakers, in recognizing the significance of these components, should prioritize allocating funds for ongoing training and mentorship programs and strengthen the institutional structures, or “hubs of services,” to make mental health services more accessible and community-centered. The fine adjustment and refinement of intervention implementation entailed knowledge transmission and resource exchange through interactive learning among multiple stakeholders in partnership, which, in turn, enhanced the delivery and outcomes of the intervention. The study findings highlight the relationship building between service professionals and the refugee community, creating a ripple effect of positive change throughout present and subsequent phases of service provision. In fact, the implementation of CB-MHPSS psychosocial groups went beyond the initial plans and expectations, leading to additional positive impacts at organizational and community levels, such as improved referrals, organizational capacity for MHPSS, community buy-ins for subsequent mental health programming, mental health sensitization, and empowerment. This suggests that collaboration with local stakeholders and tapping into their indigenous knowledge and networks is key to achieving effective and sustainable psychosocial group-based implementation. In several localities, community leaders played a pivotal role in the intervention’s success by serving as cultural consultants to affiliates and service providers, peer mentors, and interpreters for group participants. They bridged the gap between the communities, agencies, and participants and facilitated multisector collaborations that promote holistic well-being beyond the program, which is crucial for the prospect of long-term success of the intervention.
While funding is conventionally considered a primary concern for the sustainability of community programming, continued organizational and internal support, and community champions and leadership, were identified as the most critical elements to ensure continued and evolving programs. The robust capacity and capabilities of implementing and receiving agents, including participants and community cofacilitators, are equally critical to the intervention’s success, although implementer factors are most complicated and challenging to measure. Additionally, viable platforms or coordination for partnership within and across the communities, either community of practice or refugee and local communities, were essential. Linking interventions to care systems in which refugees receive less access and benefit from mental health services and system-level changes, including sustainability of community capacity building, seems most vital for long-term impact, yet it is left to local interest and resources only, creating a wide range of variances across localities and therefore fewer learning opportunities for other low-resource sites. In fact, while not directly detailed in the ToC, the outreach and engagement strategies implementers took demonstrated innovative methods for mental health prevention and promotion for underserved racial and ethnic communities, which was a pressing need that the CB-MHPSS intervention inadvertently illuminated.
While this study provides meaningful insights, it is essential to acknowledge several limitations. Initial barriers in our study involved the challenges tied to localization. A centralized and standardized evaluation approach (e.g., a structured quantitative questionnaire) was challenging to implement. For example, training approaches were not uniform and varied depending on local capacities for application and refinement. Facilitator competency methods were primarily anecdotal and based on field implementation, limiting generalizability of the findings. Second, the variability of site locations and range of conditions in which implementing sites operated limits findings. There are gaps in understanding why, how, and when modifications or adaptations occurred. While the project designers believe that a community-based group approach can be replicated with minimal cost in various settings among different cultural groups in the resettlement context, there was no clear way to directly measure cost-effectiveness. In addition, there is a lack of sufficient data for comparative study across different conditions/sites, including training, agency/community impacts and ownership, care platform or delivery flow, relations to other routine practices, and design challenges. For a more tailored fit, a systematic intervention modification of the intervention, including robust cultural adaptation methodology, may enhance the appropriateness to ensure that the intervention curriculum is compatible with participants' cultural background, values, and context [46]. Providing a comprehensive report on the analysis of a complex initiative that spans over a year in 11 states is beyond the scope of this single paper. As such, this study solely focuses on evaluating the implementation process of the MHPSS intervention groups while drawing attention to the implications of contextual factors and mechanisms on the implementation of the whole CB-MHPSS program. Finally, the ToC process is limited by its lack of significant input or revision from local partnering agencies during the visioning and implementation stages. There is a need to develop more “complexity-aware” or “complexity-sensitive” ToCs and evaluation tools [47, 48] that emphasize the interplay between complex systems and account for complexity through feedback loops or backward mapping approaches [49–51]. Additionally, uncertainty and emergence in complex evaluation settings necessitate the use of developmental evaluation methods [52], where both implementers and evaluators seek optimal solutions to strategize dynamic realities in complex systems, for example, utilizing ToC as both an outcome and a process.
Moving beyond its limitations, this study intricately explores the harmonization of facilitation goals with participant needs in CB-MHPSS interventions, providing nuanced insights into the depth of necessary adaptations [40]. By incorporating the MRC guidelines, our study meticulously navigated the complexities of evaluating a multifaceted intervention, focusing on essential components such as program theory and stakeholder engagement. The integration of the ToC framework further facilitated a nuanced exploration of the intervention’s interaction with diverse contextual factors and mechanisms, allowing for a structured and insightful analysis of qualitative data [33]. This approach has amplified the rigor of our evaluation, enabling a profound exploration of multisite implementations and underscoring the significance of community-driven strategies in developing culturally attuned interventions [56]. The study also highlights the often-overlooked yet vital roles of internal champions and community leadership in the CB-MHPSS landscape [57]. Building on these insights, we recommend that mental health professionals deepen their engagement in community-based approaches that may include comprehensive training paradigms that emphasize cultural sensitivity, trauma-informed care, and dynamic curriculum adaptation to meet the diverse needs of refugee/asylee populations. For policymakers, prioritizing resources for ongoing training, mentorship, and strengthening the foundational infrastructure of refugee resettlement programs and culturally responsive mental health services is crucial. Refugee-serving professionals are advised to prioritize community engagement, collaborate with local leaders, and proactively navigate barriers, such as digital literacy and time constraints to optimize intervention outcomes across various settings.
5. Conclusion
This study demonstrates how adopting the MRC framework facilitated a more comprehensive understanding of the complex implementation processes of CB-MHPSS interventions, providing insights into the mechanisms by which the intervention brought about a change. It underscored the importance of incorporating context, program theory, stakeholders, uncertainties, and refinement within the evaluation approach, ultimately contributing to the effective implementation and sustained improvements in the mental health and well-being of refugees and asylum seekers. The study findings highlight the critical importance of balancing fidelity to the original curriculum with cultural adaptation to ensure the effectiveness and integrity of CB-MHPSS psychosocial interventions for forcibly migrant newcomers. The key factors that facilitate and challenge successful implementation underscore the need for a deep understanding of the community’s unique needs and resources. Additionally, they highlight the importance of ongoing assessment and refinement of intervention components, allowing for contextualization based on cultural norms, the populations being served, community resources, and various other factors. The findings also emphasize the importance of collaboration with local stakeholders and the development of community capacity to achieve effective and sustainable implementation, highlighting the pivotal role that community leaders play in the success of the intervention. The agency affiliates showcased a self-driven approach to implementing interventions, while upholding a cohesive stance that aligns with strategic guidance provided by the central office in a supportive and nonintrusive manner. This study has important implications for future research, particularly in low-resource community settings, where community partnerships and local knowledge about psychosocial healing can be leveraged to achieve long-term success and community-level change. Ultimately, this study emphasizes the importance of a participatory and context-specific approach to the implementation of CB-MHPSS interventions that fully engages the community, empowers them with the knowledge and resources to lead the intervention, and results in sustained improvements in the mental health and well-being of refugees and asylum seekers.
Data Availability
The data supporting the findings of this study may be obtained from the corresponding author upon reasonable request.
Additional Points
What is known about the field. (i) Refugees and asylum seekers in the U.S. face numerous challenges in accessing mental health and psychosocial support. (ii) Mental health promotion through community-led interventions have been shown to promote emotional well-being and coping, social support, and community empowerment. (iii) While mental health and psychosocial support (MHPSS) interventions have the potential to improve the emotional well-being of refugee newcomers during resettlement, the implementation process of community-based interventions is complex and challenging, with multiple factors affecting intervention uptake and effectiveness in local contexts. What this paper adds. (i) Applying a realist evaluation approach incorporating Medical Research Council (MRC) guidelines for implementation evaluation of complex interventions, authors elaborate on implementation barriers and facilitators for a community-based MHPSS intervention, considering the interplay between the intervention and various inner and outer contextual factors. (ii) The flexibility of the psychosocial interventions to scale for multisite operations presents an additional layer of complexity, and yet it allows for adaptation and refinement, making the intervention more effective and suitable for local needs. (iii) This study emphasizes the importance of a participatory and context-specific approach to the implementation of CB-MHPSS interventions that fully engages the community, empowers them with the knowledge and resources to lead the intervention, and results in sustained improvements in the mental health and well-being of refugees and asylum seekers.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The authors thank the participants of HIAS MHPSS support groups and all the staff and facilitators across HIAS’s United States resettlement network. The authors also deeply appreciate the tremendous contributions made by Maisoon Abdelrhman, and the HIAS team: Sarmaya Mustafayeva, Camille Wathne, Mariam El Magrissy, Natalie Lam, and Alicia Wrenn. This research was possible thanks to generous support from HIAS and VCU School of Social Work (internal research grant awarded to the first author). The implementation of the project was possible thanks to a grant by a private foundation fund, Crown Family Philanthropies.