Research Article

Implementation Evaluation of Community-Based Mental Health and Psychosocial Support Intervention for Refugee Newcomers in the United States

Table 3

Template analysis of the multilevel implementation process.

MultilevelProcess by ToCImplementationRefinementImpact
MechanismOutcomeFacilitatorsChallenges (what did not go well, barriers)AdaptationOutcomes

Intervention level wellness improvement(i) PA: HIAS authors a tailored, culturally-appropriate curriculum specifically designed to promote wellness amongst newcomer communities that community facilitators deliver with affiliate support
(ii) PB: participants interact with curriculum delivered by community facilitator over a 9-10 week period in a confidential and safe space (remote or live format)
(i) Outcomes participant-level 1 (OP1): participants will be more equipped with new MHPSS skills, strategies, and social networks
(ii) OP2: participants experience behavioral change and have improved perceptions of mental health and psychosocial wellness and increased knowledge of local resources
Ex. participants will have an increase in knowledge and awareness of local MHPSS clinical and non-clinical resources
(iii) OP3: participants will have an improvement in the mental wellbeing (defined as the emotional and daily functioning aspects of mental wellbeing) and perceptions
Ex. improved short-term wellness, improved long-term wellness practices, improved ability to seek additional support (Warwick+ HIAS 5 item scale)
(i) Supplementary activities and resources: “their Spanish group has been finding TTT tapping videos online, in Spanish, and they have been well received by participants. A staffer shares, “one of my participants says that every time we do one of these activities, she loves them and wants her children to do them too.” (locality J)
(ii) Experiential learning: “their group members appear to really enjoy the activities introduced. One staff member, Stella, commented, “the closing activities that we use to close the sessions are more often than not, the highlight of the session.” (locality A)
(iii) Effective exercise and activities: “I was surprised by how effective the activities have been…participants seem to respond to activities really well.” (locality J)
(iv) Difficult topics (trauma): “they state that the session on trauma (session 7) was quite difficult for many, but by the time they arrived at the closing activity (body scan exercise), many participants appeared more grounded and ready to leave the session calmly.” (locality B)
“reports their participants seemed to respond well to the curriculum as it is, especially the “river and the bridge” activity in the final session” (locality A)
“Facilitators indicated that the best weeks of the curriculum included traditions and healthy living; coping techniques; finger tapping/body scan; and weeks 7 and 8 related to trauma and the future.” (locality H)
(v) Engagement: “the facilitator shares that participants are very engaged and are most interested by sessions that offered a lot of “skills” or “take- home activities” that allowed participants to learn new things.” (locality D)
Group rapport: “Our groups have built such strong rapport now, which took several sessions. It’s unfortunate that the program is only 9 weeks. We definitely want to continue these groups in some capacity in the future.” (locality I)
Social bonding, building community relations: “Doing this group in the same language and same community has made a big difference. This is the first time for some of our clients that they are able to bond with people from their home country, discuss traditions, and speak in their own language.” (locality F)
“The biggest lesson learned was how beneficial it is for the group members to process trauma with others who have similar experiences as their own.” (locality K)
“Both groups admitted that coming together as a group every week made them feel supported and created a space to talk about issues and problems they usually do not discuss with anyone. They were happy to learn from each other’s experience and to listen to suggestion and solution others discussed during these sessions.” (locality C)
Virtual/remote: “it was great for all the attendees to join virtually, as they lived all over the state the of Delaware as well as a few in other states. Clients stated they felt supported, laughed, played games, shared their stories and were able to work on their wellness by attending these groups.” (locality F)
“Benefits of remote programming included more participation than we have seen in past in-person groups…we also were able to provide meaningful digital literacy training and help participants improve their technological self-sufficiency…It was evident that many group participants have experienced a great deal of social isolation due to the COVID-19 pandemic and lacked an outlet to discuss the challenges of this past year.” (locality D)
Gained skills: “One youth member told us that “this group has helped me learn a lot of ways to manage my stress and remain positive in difficult situations. I would like to be a part of another group like this in the future.” (locality K)
(i) Heterogeneity of group participants (finding commonalities between group members): “they had some challenges with finding content that resonates with all group members; they face unique challenges here, as the only defining characteristic for their groups is language and immigration status. Mentors gave great ideas here.” (locality J)
Difficult topics: “they state that the session on trauma for their youth group was particularly difficult, but they navigated their way through it the best they could.” (locality A)
Not enough time: “I feel as if we are finally getting over this hump of getting to know one another, and we’re finally in a space where we want to be, but now it’s ending. We’re longing for a few extra sessions.” (locality J)
Curriculum challenges, incl. pre/post surveys: “the pre and post surveys are effective but too cumbersome. It would be enough to have one of each, not two. Also, we would like to have them translated for future use.” (locality E)
“Reports by some groups that they felt the Covid aspect of the curriculum was out of date to the current circumstances surrounding pandemic.” (locality H)
“There were many women who ignored our emails and reminders and later confessed to me that they found the survey to be incredibly confusing and were worried that they would get in trouble if they did not complete it. This was what made us decide to forgo implementing the post survey.” (locality J)
“We did not use pre/post-survey B as the restrictiveness of the questions and the way in which they are phrased would not have worked well with the participants of our groups. Instead, we used survey A and found it to be beneficial.” (locality G)
“We were not able to conduct the pre and post survey with every participant due to limited capacity, inconsistent attendance and literacy barriers.” (locality D)
Curriculum appropriateness: “some topics, such as nutrition can be very difficult to discuss with group members due to cultural or economic reasons…While most feedback were very positive, group facilitators reported feeling uncomfortable when addressing nutrition within the group because of the differences in diet between members.” (locality K)
“It was challenging to adapt feedback and advice in the context of supporting asylum seekers and non-refugee status immigrants…this was really designed for refugees. Our groups were very diverse and composed of women from completely different countries, continents, religions, etc.” (locality J)
“In some cases, our facilitators found the language in the curriculum was not easy to translate into languages in which our groups were conducted. Some of the concepts outlined did not align with cultural values or perceptions of mental health. Notably, one of our groups elected to skip the session on healthy living because they felt the curriculum did not take into a full account the relationship between food access, nutrition and culture.” (locality D)
Scheduling conflicts: “only one group of young Bhutanese men did not continue past week 2 due to various scheduling conflicts among the members.” (locality H)
“participants reported challenges in attending weekly meetings for the duration of 8 weeks citing conflicts with their prior commitments.” (locality K)
Group size: “the only feedback I would suggest in terms of structure of the mentorship would be to have smaller group zooms to encourage more depth of discussion and sharing. There were many of us in the group which I think made participants more hesitant to share.” (locality J)
Group rule adherence: “we also found that we needed to be flexible with some of the established group rules…If we strictly enforced this group rule each week we would have significantly limited group attendance.” (locality B)
(i) Cultural adaptation of curriculum: “this group shared a very interesting modification that is not recorded in the weekly fidelity surveys; for Eid last week, the groups did not continue with the planned session but instead added a separate session that allowed the group to still have weekly sessions, while only celebrating Eid together. (A deviation from the 9-week curriculum suited to a particular culture and community that may be valuable to capture)” (locality D)
(ii) “Locality K is modifying the curriculum to address a recent loss of a youth community-member and subsequent bereavement needs. After discussion with Camille and Sasha, they will instead create an immediate two-week “youth wellness camp”, still following the curriculum themes and weekly sessions, but on a weekly basis.” (locality K)
“Added some additional self-care activities before closing their groups, due to participant anxiety about the support groups ending. They gave out certificates and hope for future programming” (locality I)
“The cofacilitators were encouraged not to divert too far from the curriculum but also given the space to be innovative with how they presented each session and discussed the various topics. This led to more productive discussions among all group participants.” (locality F)
“We learned to continuously adjust how we implement the programming and offer resources within their range of comfortability.” (locality G)
(iii) Additional relationship building: “as a result of their in-person convening after a long period of isolation, their group is diverging a little bit from the curriculum weekly themes, and are instead spending a lot of time on rebuilding relationships in-person. Bashar states that they have needed this diversion from the core curriculum content, in order to support necessary rapport-building.” (locality C)
“She did follow the curriculum “a lot” for the first three sessions, because “I needed to establish a reason why we are in the support group.” Then, she brought in more of her own exercises and ideas, still while following the core curriculum themes and discussions each week.” (locality F)
Participants as co-facilitators: “in one group for Spanish-speakers, their team struggled to find any facilitator, back in April. So instead, they’ve had their staff-member Emily facilitate with a different participant each week–they say it has ramped up buy-in into the group, and participants are really liking this model of changing up the facilitation.” (locality D)
Group continuity beyond the intervention: “the participants state that they are setting a date to keep meeting as a support group at a Nepali community space in Springfield, every Saturday.” (locality I)
“Their members are finding ways to connect via WhatsApp, but still there is sadness to close those groups out formally.” (locality F)
Group and community appreciations: “last week they decided to do an affirmation circle wherein each participation stated something they appreciate about another, and then participants stated appreciations about their community facilitator.” (locality J)
Connection to ongoing care: “two participants in the group were linked with our center for Survivors of Torture program for ongoing support and clinical care. This was a direct result of the group experience and learning.” (locality E)
Exchange stories, talk about things out of community norm: “other groups reported that the meetings were a meaningful way to meet together and also appreciated having the opportunity to talk about topics from the curriculum that are not usually part of the norm in their community groups and was a chance to exchange stories about their past and current experiences.” (locality H)

Affiliate level stronger relationships and knowledge increase regarding MHPSS services (focused and non-focused)(i) AA: training and mentoring
(a) HIAS training series delivered (3-part, +1 safety guidance)
(b) Training by affiliates to community facilitators
(c) Mentorship Bi-weekly discussions

(ii) AB: resources, guidance
(a) Use of guidance’s, and training annex developed by HIAS, service mapping, etc.
(b) Mentor roles defined and discussed
(iii) AC: fidelity, monitoring
(a) Final listening session recorded
(b) Notes of Bi-weekly discussions
(iv) AD. Outreach and relationships
(a) Recruitment of community facilitators (listening sessions) by affiliates
(b) Training by affiliates to community facilitators
(i) outcome affiliate 1 (OA1): HIAS MHPSS training and implementation components will result in an improved capacity of HIAS affiliates and community leaders to identify and refer at-risk clients for additional mental health support within the local mental health landscape, throughout and after the program cycle
(ii) OA2: affiliates will be equipped to plan and implement mhpss support groups, and train community leaders
(iii) OA3: affiliates and community leaders co-implement the 9-10 week mhpss support group curriculum
(iv) OA4: affiliates and community leaders have greater knowledge of MHPSS principles, local MHPSS resources, target local community needs, as well as increased affiliate network contacts and resources
(i) Innovations from less structured, small mentoring groups (more flexibility)–locality E
“This group’s mentors are leading with perhaps less structure and less of a discussion outline, as opposed to other mentorship groups. This has led to a lot more excellent fluid dialog, excellent suggestions, amazing tangential tracks, but also a few silences, because it’s a slightly smaller group. Mentors continue to have great remarks, but do not lead with a formally structured style, so there is some slight confusion/silences.” (locality C)
(ii) ”They have identified a facilitator, but have not yet begun recruitment. Their planning sounds extremely thoughtful and includes a lot of curriculum training.” (locality E)
Attendance/retention strategies: “they have somehow managed to maintain attendance at 24 every week, with only a few dropping in and out every week. Their facilitators report excellent and ongoing outreach.” (locality I)
“They continue participant engagement tactics to ensure attendance is robust, each week.” (locality G)
“They are planning to offer a gift card or financial incentives for attendees who make it to most sessions, which is inventive.” (locality C)
“We were also able to encourage attendance by providing a $50 gift card incentive for those who attended at least 50% of the meetings. In-meeting participation was encouraged by calling on participants directly and encouraging them to participate in the discussion.” (locality B)
“We encouraged participation by building rapport, providing Chromebooks and wifi stipends, offering financial incentives, soliciting feedback and in the case of one group, invited participants to support with facilitation. Throughout the program we sent weekly reminders and followed up on an individual basis as needed.” (locality D)
“We provided significant incentives for participating in the MHPSS program, including giving each participant $15 for every session they attended and allowing participants to keep their tablets at the end of the program.” (locality I)
Peer mentorship/staff meetings: “JFS staff who were new to doing this program stated they appreciated the biweekly meetings we had with other agencies who were implementing the program, they stated it allowed them to bounce ideas and solutions off of each other which was very helpful.” (locality A)
“Peer mentorship was useful as it allowed us to ask questions of people who had done the work and share successes, challenges, and ideas.” (locality E)
“We found the affiliate peer mentorship structure helpful. The meetings were a good way to learn about what was happening in various sites and provided a good platform to share good practices.” (locality F)
Recruitment: “the best form of advertisement to get attendees were from referrals of other active participants in the group.” (locality F)
“We encouraged participation by describing this as a women’s support group and not as a mental health group or anything mental-health related. We were very specific in calling this the HIAS women’s group, given the stigma surrounding “mental health” spaces and groups.” (locality J)
(i) Delays in implementation: “for their other two groups, they’re not fully fleshed out yet and they’re still in the process of facilitator training, and funding and interpretation. They may be relying both on a heavy interpretation model and a heavy affiliate staff involvement.” (locality E)
(ii) ”Flagging some fairly high concerns here; the groups have not begun yet and they are unlikely to begin soon. It looks like their group at the Women’s Shelter is not working out for various reasons they shared, and they are transitioning away from the idea of hosting a support group in that shelter. They are transitioning to outreach within two other new refugee communities, but they are only now beginning communications to facilitators and new participants with nothing on the calendar. It is highly unlikely that they will meet the project timeline, per their words; they state they may be ready for implementation in a month or two.” (locality E)
(iii) Low recruitment: “they have low recruitment for their third group but everyone shared great solutions for recruitment & outreach.” (locality K)
(iv) Recruitment timing during holidays: “as we are aware, will likely have to start implementation post-Ramadan in mid-May due to universal recruitment challenges. I just caught your note about being transparent with others about their late timeline and I really need to figure this out because it’s a bit of a tension.” (locality C)
(v)”Ann Arbor has been transparent about the late implementation of this group, due to inability to convene the group during Ramadan.” (locality A)
(vi) Budget: “everyone is struggling to maximize the budget and thought it was a bit low, but this is because so many of them are having to invest in remote considerations and pilot considerations – like devices, or first trainings for first community facilitators.” (all, 4/14/21)
Attendance/retention: “they’re going well but their attendance is low and they are doing extra outreach to try to solve this.” (locality F)
“Groups are going well but with low attendance.” (San Diego)
“The Spanish-speaking parents group & the Congolese women’s group are going well, but with similarly low attendance.” (locality D)
“Some groups have attendance of 1 person per week, and the next week it is 5 persons; they are learning to adjust each week.” (locality D)
Online programming: “online programing for the youth population is particularly challenging. For youth, in person programming would work better.” (locality A)
Content heavy: “there were times when the weekly sessions were too content heavy and could become more like a lecture if we tried to incorporate everything. Additionally, most of our sessions were limited to 1 hour, so we were not always able to incorporate all of the activities and content each week; it would be helpful to condense the curriculum, as 1 hour is typically easier for people to commit to as opposed to 1.5 hours.” (locality A)
“Some facilitators felt that in certain weeks of the curriculum that they were acting more as lecturers to the topics.” (locality H)
“A few of the sessions seemed to have a bit too much content packed into them, making it difficult to complete the session in the allotted time, while also ensuring that all the topics were covered thoroughly and that individuals had a chance to contribute to the discussion meaningfully. Specifically, module 2C was quite vast and required more time engagement than what was allotted.” (locality I)
“We believe that some sessions are too long, and many topics cannot be covered in one and a half or even two hours.” (locality C)
Mentoring go beyond and above: “there is a lot of inter-group advice being given, and not just mentors to mentee. They’re all sharing from all their past experiences with mental health, client outreach, and other tips and experiences that absolutely relate to this program–a lot of pulling in from other program learnings. Everyone continues to provide stellar advising to most problems. (locality E)
(i) Mental health tools: “buffalo shared a genius idea that they will give out all crisis line numbers in the region to participants by the end of the group. Other affiliates loved this.” (locality E)
(ii)”They have a licensed clinician and staff-member, supervising three un-licensed MSW staff members, who are supervising or co-facilitating groups with a community member…it really shines through in their organized approach and ability to introduce diverse mental health tools.” (locality A)
Learning and sharing beyond intervention/program: “interestingly, all 3 mentorship groups continue to use these spaces as opportunities to update staff, from their own affiliates, on program updates. For example, “by the way, co-staffer, I received the Chromebooks,” or “by the way, supervisor, I recruited all the participants, and we need to discuss this” So fascinating to witness! (Locality E)
(iii) Clinical expertise suggestions: “they encouraged that if any sites have a therapist on-staff, that another option to continue MHPSS groups is to have them led by an in-house therapist, and maybe the therapist can bill for the group. Another wonderful suggestion from their depth of expertise in the clinical realm.” (locality A)
Incorporate MHPSS into case management: “As individual practitioners, we hope to incorporate MHPSS tools and resources into our practices with clients through case management and incorporate MHPSS skills with other topics that are relevant to our clients.” (locality A)
Importance of creating trust: “I think that we learned a lot of lessons about creating space for connection and trust. Especially over zoom, that does not happen naturally.” (locality J)

Community level knowledge, (skills and competencies) increased with regards to MHPSS focused and non-focused services(i) CA: community facilitators teach curriculum, facilitate content related to coping strategies, building family and community-resilience, cultural adaptation, trauma healing, and other MHPSS topics to target refugee community
(ii) CB: weekly fidelity checklist
(iii) cc: training by affiliates to facilitators
(iv) NOTE: nothing used to measure competency or knowledge of facilitator
(i) OC1: community leaders are equipped to identify at-risk clients and referral resources
(ii) OC2: community members are a part of the local structure that can sustain MHPSS programming and address the mental health needs of the local population through clinical and non-clinical resources
Participant attendance and engagement: “this is remarkable, but they have somehow managed to maintain 24 attendees every single week. I can only guess here, but I think part of what made their groups so successful is it seems their two community facilitators and staff-members –Durga and Reeta–are deeply embedded in the Nepali Bhutanese community and are stellar community leaders” (locality I)
Community strengthening bonds: “they share that they are witnessing community-bonding that has crossed divisions, because people are bonding on the basis of shared home-culture or hispanic identity. They share surprises of connection and support in-between people who are vastly different.” (locality K)
“My participants have a community that they can now rely upon to help them navigate any challenges or any barriers that happen, now,” (locality A)
Facilitator mental health improvement: “we have worked with some of the facilitators for years, and suddenly we’re hearing them say they feel like themselves again, that they haven’t felt like themselves since they’ve come to the US, but in this group, they feel like a leader again. We’re learning that they were human rights defenders and prominent leaders in their home country, and they haven’t felt like their former selves until they started this group.” (locality K)
“One newcomer Leader stated that prior to her participation in this group she had lost her leadership skills and felt that a part of her was broken. However, through her participation and leadership of this group she shared that she finally is starting to feel like herself again and will continue to practice the skills learned from the peer support group in her everyday life…Even though the Leaders were processing their own trauma, immigrant experiences, and daily struggles, they were excited to lead the group and provide support to the other participating members. This has been a lesson to all participating staff because it shows the motivation the group leaders have to help others, as well as themselves during the adaption process they are undergoing.” (locality K)
Rapport building: “the training for community members and subsequent outreach conversations have begun to build the trust needed to effectively address impacts of trauma and mental health needs.” (locality E)
Recruitment: “CRRA staff recruited former clients from various language groups (Arabic, Swahili and Nepali) to act as facilitators for this project. The facilitators, in turn, recruited participants from their respective communities who were interested and willing to join the 9 week program. Because the facilitators were well entrenched in their communities, the recruitment and maintenance of participation levels was sustained throughout the project.” (locality H)
Facilitator insider knowledge of community: “Facilitator was leading members from the same exact national background. She felt this created a level of comfort and understanding that allowed her to generally lead the Support Group, but even more confidence to bring in Performing Arts techniques, due to her understanding of what would and would not be appropriate with refugee from her same country.” (Locality F)
“It was helpful to have facilitators from the resettled refugee communities themselves. We found this to be important because the facilitators not only spoke the language and could deliver the sessions in the local language (i.e. Nepali), but they also had a shared resettlement experience and could relate closely to those in the group. We think given cultural dynamics, this was important to foster trust and openness. Furthermore, the facilitators that were selected are well-known and well-liked by the community, which also likely fostered trust and engagement among participants with the program.” (locality I)
Uncomfortable with co-facilitator role: “many of the refugees we served who we asked to be co-facilitators stated they did not feel comfortable in this role.” (locality A)
Lacking cultural knowledge: “we learned that people are looking for help but don’t know what they need or how to access services. Also, these topics are complicated by language and cultural dynamics that we are beginning to understand on a deeper level” (locality E)
“One recommendation would be to invest in audio or video exercises in additional languages. community co-facilitators did not always feel comfortable leading certain exercises and having audio or video resources to reference in these instances would be very valuable.” (locality B)
“Another issue is that co-facilitators sometimes do not speak English, so we would spend most of the time translating the curriculum instead of talking about what we will be doing during the session.” (locality C)
(i) Group participant feedback on mental health: “they took feedback from the Pilot on participants wanting to know about mental health and how now built a new module in R&P cultural orientation about mental health –they are recommending that New Sites do this, too.” (Locality A)
Levels of staff safety support: “they have four levels of staff safety support: a primary community facilitator, an alternate community facilitator, an affiliate case manager who attends all sessions (who does not participate or intervene, only witnesses), and a supervisor for each group.” (locality K)
“One benefit of completing the MHPSS program was that it gave us an opportunity to review emergency safety procedures for those experiencing a crisis.” (locality B)
Professional mental health counselor support/connections: “during the youth listening session, staff learned of recent tragic loss experienced by the youth group. Although the youth group members emerged as potential leaders, staff felt the need to address the recent traumatic loss experienced and invited a professional mental health counselor to assist with the group. The counselor helped facilitate the sessions and incorporated coping mechanisms in dealing with loss and grief throughout the 8 weeks curriculum.” (locality K)
“Engaging in the community resource mapping prior to programming also allowed us to develop new contacts for mental health referrals.” (locality B)
“We are planning to have a mental health counselor in our office who speaks the clients’ language to mitigate any misunderstandings or frustration usually caused by translation. We have been partnering with different organizations in Toledo and created excellent partnerships with them.” (locality C)
Additional training for facilitators: “the newcomer leaders will participate in a mental health first aid training and will receive a certificate upon completion. This training will further prepare them to continue their role as leaders in their community.” (locality K)
Guest speakers: “lastly, we found it very meaningful to have an opportunity to bring in a guest speaker. Our guest speaker was a trusted community member who has a diverse knowledge of the practices of different religions and subgroups with the Bhutanese community. The group participants seemed to enjoy having a guest speaker share his knowledge.” (locality I)