Abstract

Supporting family caregivers has been identified as a high-priority health goal worldwide. To reach this group, low-threshold, flexible, and accessible support services are needed that are individually adapted to the needs of the affected families. The newly developed digital counseling service “Alles Clara” intends to provide support for this group by connecting them with healthcare experts, allowing for professional and individual counseling at any time and place. To demonstrate the impact of Alles Clara and to be able to further develop and improve the service, it is to be evaluated. To uncover what works, how, under which conditions, and for whom, we drew on the realist evaluation methodology to devise an initial program theory. In this article, we describe the development of and present the initial program theory of Alles Clara. Internal program documents and guided individual interviews with stakeholders and counselors served as data sources to gradually develop a preliminary understanding of Alles Clara’s intended impact and modes of action in an iterative process. The initial program theory outlines central chains of events, mechanisms, and outcomes of the digital service. Family caregivers experience relief through expanding awareness, appreciation, trusting relationships, and reflection. For professionals, a new professional environment, a healthful culture, quality, and meaningfulness affect job satisfaction. Central to unfolding its mechanisms is that the digital counseling service enables an experience of “being seen” while “not being seen.” The results provide information about the impact of Alles Clara on both the target groups, identify possible areas for service improvement, and serve as a basis for evaluation and regular monitoring, especially, when it comes to the question of whether and how digital services can be supportive in the sense of a person-centred orientation of health care.

1. Introduction

Over a third of the European population provides informal care [1]. In Austria, around 10 percent of people are involved in the care and support of a person close to them [2]. Family caregiving is considered a key pillar of healthcare delivery, enabling people in need of care to remain in their homes as long as possible and to maintain their quality of life in the best possible way [3]. Demographic trends indicate that the number of people in need of care and the need for informal care by their relatives will increase in the future, which is why supporting and strengthening this group has been defined as a high-priority health goal worldwide [46].

Almost 50% of family caregivers in Austria feel strongly burdened by their responsibilities [7]. Lack of financial resources and time as well as physical and psychological stress are described as antecedents of caregiver burden [3, 8]. One of the biggest influencing factors is lack of knowledge and information [9]. Despite the great need for support, family caregivers remain largely invisible and inadequately perceived in their role [2]. To reach family caregivers, there is a need for low-threshold, flexible, and accessible support services that are individually adapted to the needs of the affected families [5, 9].

The World Health Organization [6] recommends, among other approaches, the use of information and communication technologies to support family caregivers. Although digital solutions are now increasingly used in formal care settings, they are still less considered in family care, although a high potential for relieving the burden is expected there as well [4, 10, 11]. Electronic health (eHealth) interventions, especially low-threshold and needs-oriented services, can contribute to supporting family caregivers [12, 13].

With the aim of relieving the burden on family caregivers by using digitization, a new social service was launched in Austria. Alles Clara connects family caregivers with professional counselors in virtual rooms of a data-secure tool, the Alles Clara app. The digital service thus intends to enable professional, low-threshold, flexible, and personal counseling, independent of time and place, via messenger. Thus, in Alles Clara, communication is mainly text-based, in the form of an asynchronous chat. This way, family caregivers have the option of re-reading the conversation at any time. Counseling is provided by certified nurses, psychotherapists, and clinical and health psychologists, qualified as online counselors. The topics of the consultation are not predefined but are based on the concerns of those seeking advice, regardless of the care situation they find themselves in (care for adults, children, acute, long-term conditions, end of life, physical, or mental healthcare needs). Alles Clara serves as a digital interface between the two user groups and links closely to Austrian healthcare services. The goal is to offer family caregivers security and support in their daily lives and to refer them to existing, analogue services. In addition, the developers of Alles Clara intended to offer healthcare professionals employed with Austrian healthcare organizations a new field of activity, characterized by flexible working conditions and client-distant operation.

Alles Clara was launched in July 2022. During this pilot phase, employees of selected Austrian companies can already use the digital service free of charge. This allows facile access to a heterogeneous user group. In order to demonstrate the impact of Alles Clara and to be able to further develop and improve the social service, it will be evaluated during the pilot phase.

The challenge in evaluating such a service lies in its complex nature. Complex interventions consist of several individual components that act either independently or interdependently in a specific context [14]. To date, it is poorly understood how app-based counseling and support services such as Alles Clara work, as common evaluation measures for eHealth interventions, primarily focus on usability and acceptance of the technology but often fail to detect its efficacy and impact [15]. Ruggiano et al. [16] recommend relying on conceptual models to inform the development of mobile apps. Sala-Gonzáles et al. [13] describe the need to uncover “key elements” of the interventions to guide eHealth evaluation studies. Grounding an intervention in a conceptual and comprehensive theoretical framework is increasingly used and discussed in nursing and health intervention research [17]. This informs operational procedures [18] and provides the possibility to capture and verify the actual impact of Alles Clara in its complexity.

Therefore, we employed the theory-based evaluation approach of realist evaluation (RE), aligning evaluative measures with a prior theoretical understanding about modes and chains of effects. This inquiry is guided by the question “What works for whom in what circumstances and in what respects, and how?” [19]. In this paper, we describe the development of an initial program theory of Alles Clara to go into depth and gain a precise understanding of its intended impact as a first step within the process of realist evaluation.

2. Materials and Methods

2.1. Methodological Approach

RE represents an approach to theory-based evaluation and is grounded in the realist philosophy of science [20]. Theory-based evaluation allows us to understand not only whether a program works but also how it works, by theorizing and modelling the path from an intervention to its intended change [21, 22]. The core of a theory-based evaluation thus represents the assumption of a logical model of how an intervention contributes to a chain of intermediate results and finally to the intended outcomes [22]. With their conception of RE Pawson and Tilley [23] proposed a methodological approach assuming that an intervention per se is not what works; rather it becomes effective (or not effective) because actors make (or do not make) certain decisions in response to the intervention. Actors’ “reasoning” in their response to potential resources or opportunities provided by the intervention is described as the cause of outcomes [24]. It is assumed that an implicit theory underlies the mode of action of such interventions [25].

To uncover this implicit theory, RE relies on three key concepts that guide hypothesis generation and formulation. Through the intervention, specific “mechanisms” (M) are activated in a particular “context” (C), leading to intended and unintended “outcomes” (O) [26]. The underlying cause-effect relationships are formulated as context-mechanism-outcome “configuration” (CMOc). Developing an initial program theory is the first step in identifying these mechanisms by which the intervention produces particular outcomes in particular contexts [27].

2.2. Program Theory Development
2.2.1. Gaining a Preliminary Understanding of the Program

We based the development of the initial program theory of Alles Clara on three data sources, to involve different perspectives. These included (1) internal program documents, guided individual interviews with (2) stakeholders (S) and (3) counselors of Alles Clara (AC), and served to gradually develop a preliminary understanding of Alles Clara’s intended impact and modes of action in an iterative process. This preliminary understanding laid the foundation for the theory development process (see Figure 1).(1)Program documentsThe aim of the document review was to draw conclusions about the theoretical embedding of the program via the described development process, goals, and principles pursued. We analyzed internal program documents such as documentation on project planning and training materials of the counselors, which enabled the development of a preliminary understanding of the project. In addition, the information gained from the documents served as a reference point for the aspects that needed to be explored in more detail in interviews with key informants.(2)Stakeholder interviewsSubsequently, we conducted an exploratory qualitative study of stakeholders’ assumptions and perspectives of Alles Clara counselors on their work within the program. Nine guided individual interviews were conducted with stakeholders who were actively involved in the development of Alles Clara (i.e., members of the business and program development teams and project partners).(3)Counselor interviewsAssumptions derived from program documents and stakeholder interviews formed the basis for these interviews. We interviewed five counselors of Alles Clara, who are experts in the fields of nursing and psychology and have many years of relevant work experience. Interviews focused on their understanding of their role in Alles Clara, their experiences in counseling, and their expectations regarding the impact of the program.

We analyzed data thematically [28]. Data indexing focused on identifying accounts of how outcome patterns are formed by mechanisms and context. This enabled the development of a deeper preliminary understanding of the program and its modes of action.

2.2.2. Constructing the Program Theory

As an essential part in the process of theory development and to finally answer the underlying question “What works for whom, under what circumstances, in what ways, and how?” we conducted iterative discussion rounds within the research team.

Prior to formulating CMOcs, we explicated the contextual circumstances in a structured manner. For this purpose, we considered the categories of theory of action (ToA) according to Chen’s [25] program theory for data analysis, which is informed by the intervention protocol of a complex intervention. We analyzed and abstracted the statements of the counselors regarding their actions and procedures by means of a thought experiment [29] in order to identify the core intervention of Alles Clara. In an iterative process of critical reflection and discussion guided by retroductive reasoning [30], we developed a logic model, an initial program theory, relying on the heuristic of CMOcs. In this process, we used the questions regarding CMO generation [31] as a guideline. Based on identified outcomes, we uncovered mechanisms and causal relationships. We structured assumptions about causes and mechanisms of action with the help of graphical representation and formulated CMOcs to explicate the results as testable hypotheses.

2.3. Ethics

This study is part of a larger project, which has received ethics clearance from the Karl Landsteiner University of Health Sciences Ethics Committee (registration number 1082/2020).

3. Findings

The initial program theory demonstrates the intended outcomes of the intervention and reveals the mechanisms and context that may lead to the impact of Alles Clara. The theory consists of multiple CMOcs.

The primary target group is individuals who care for someone close to them. Alles Clara intends to relieve the burden of family caregivers. We furthermore identified mechanisms that affect the counselors who work at Alles Clara. The initial program theory of Alles Clara therefore consists of the following three levels of impact:(1)The impact of the Alles Clara app on the family caregivers(2)The impact of the counseling on the family caregivers(3)The impact of Alles Clara as a working environment on counselors

At these levels, we identified various mechanisms that lead to relieving the burden on family caregivers and increasing the job satisfaction of Alles Clara counselors.

3.1. The Impact of the Alles Clara App on Family Caregivers

We identified different mechanisms that have an impact on the family caregivers in Alles Clara and ultimately lead to them registering in the app and “carrying” Alles Clara with them as a constant companion in their pocket (see Table 1). The design, language, color scheme, functionality, and low-threshold accessibility were defined as mechanisms for those seeking advice feeling comfortable, addressed, and secure. After registration, family caregivers have the Alles Clara app, and thus the opportunity to get in touch with professional counselors, in their pocket. This relieves the burden on family caregivers because they no longer feel alone.

3.2. The Impact of Counseling on Family Caregivers

The core of the Alles Clara program is the professional counseling for family caregivers. In the counseling, people seeking advice are given the opportunity to (a) reflect on themselves, (b) discover new ways of coping with their situation, (c) experience appreciation, and (d) build a trusting relationship with the counselor. This way, further mechanisms of action unfold on a second level of impact, which leads to the people seeking advice experiencing relief (Table 2).

The counseling begins with the person seeking advice describing their current question, challenge, or situation. In the process of writing, the family caregivers are becoming more aware of their situation and get it off their chest. Writing down the questions and challenges can initiate a feeling of relief. The interaction with the counselors initially leads to a development of awareness. New possibilities open up for the family caregivers to deal with stressful situations that were not apparent to them before. In addition, people seeking advice experience appreciation and thus develop a trusting relationship with the counselors.

Through this process, people seeking advice are enabled to find new ways of coping with their situations and challenges and address emotional issues. The counselors succeed in encouraging and empowering those seeking advice to cope with challenging situations. The interplay of these mechanisms results in relieving family caregivers.

3.3. The Impact of Alles Clara as a Working Environment on Counselors

Alles Clara cooperates with large Austrian care organizations (NGOs) and enables the employees, who are mainly nurses, to experience a flexible element in their professional lives through Alles Clara. The counseling activity complements the existing employment of the counselors. Working at Alles Clara leads to increased job satisfaction for counselors. The professional environment is characterized by a healthful culture and provides opportunities to organize work processes flexibly and individually (see Table 3).

The counselors have time to deal with the concerns of the family caregivers, to review literature and textbooks, and to structure and carefully draft their messages. In addition, they bring their specific professional expertise and experience to online counseling and are equipped with further knowledge through their work in Alles Clara. The counselors in Alles Clara share specific values, which guide them in their encounter with the people seeking advice. Knowledge, time, and their attitude enable counselors to provide high-quality counseling. By perceiving the impact of their work on family caregivers, the counselors recognize the meaningfulness of their actions. Recognizing and being able to influence the quality and meaningfulness of their work has a positive effect on job satisfaction. This way, the nursing profession in particular benefits from working at Alles Clara.

3.4. Summary of the CMOcs

The CMOcs describe the intended impact and how it is created by Alles Clara for both the target groups. While it is important to articulate CMOcs as a written text on the different levels on which the program unfolds its impact, it adds value to see how these single configurations come together as a whole. Constructing a configurational map (see Figure 2) allows to demonstrate the interplay between mechanisms and to represent the logic of the program. Findings indicate that in the context of digital counseling, flexibility and a low access threshold are key features, and the aspect of anonymity plays an important role. The fact that family caregivers are not seen in digital space encourages them to turn to the counselors with emotional questions. People seeking advice and counselors enter a trusting relationship, thereby opening up to each other on a personal level. Counselors see the persons, their situation, efforts, and capabilities. Appreciation and empowerment are the central mechanisms for relieving the burden on family caregivers and can lead to increased job satisfaction of counselors. The interaction of all these components characterizes Alles Clara as a program and conditions its effectiveness. This interplay reveals an indirect impact on a societal level, as the implementation of Alles Clara may affect the perspective on nursing in Austria.

4. Discussion

The initial program theory provides insights into how the Alles Clara program intends to achieve beneficial outcomes for family caregivers seeking advice and counselors by outlining central chains of events, mechanisms, and outcomes. Family caregivers experience relief through interventions and mechanisms that work through expanding awareness, appreciation, trusting relationships, and reflection. For counselors, the program provides a new professional environment, characterized by a healthful culture. Quality and meaningfulness of their actions affect job satisfaction.

Since the early 2000s, as the use of technology and digitization in everyday life has increased, so has the development and use of eHealth services [13, 32]. Digitization and technical assistance can be used in a variety of ways and offer a high communicative potential [4]. Implementing technology in health care can be problematic; however, certain digital tools have shown potential to improve a person’s health outcomes and also be of benefit on the part of healthcare workers [33]. The goal of this inquiry was to investigate whether and how Alles Clara, as a digital service, can impact family caregivers by using program theory to develop a deeper understanding of the intervention. Alles Clara, like other existing services [13, 15], aims to relieve the burden on family caregivers by using digitization. However, program theory not only shows the intended effect of an intervention, but also reveals the mechanisms that lead to this effect. Yet, the process of developing a program theory can be fraught with challenges.

Giel [34] describes a clear gap between the conceptual considerations and the methodological approach to program theories. For instance, challenges associated with identifying and mapping the key concepts (context, mechanism, and outcomes) are repeatedly emphasized in relation to the realist evaluation approach [35, 36]. Distinguishing implicit program mechanisms from explicit program properties is described as one of the major struggles when developing CMOcs [35, 37]. To address these challenges, we linked Pawson and Tilley’s [23] realist evaluation approach with Chen’s [38] understanding of program theory, more specifically the theory of action as a method to identify the circumstances without which the phenomena could not occur. While these different methodological approaches rely on different concepts and heuristics, they share a common epistemological basis in the realist paradigm [39]. As theory development relies on the process of retroduction, which entails “building models using cognitive material and operating within the ambit of analogy and metaphor to uncover structures and mechanism” [40], we used thought experiments as proposed by Jaccard and Jacoby [29] to identify variable relationships.

In Alles Clara, family caregivers experience relief through interventions and mechanisms that work through expanding awareness, appreciation, trusting relationships, and reflection. For professionals, a new professional environment, a healthful culture, quality, and meaningfulness of their work lead to job satisfaction. Alles Clara, as a digital tool, is characterized by a peculiarity, which also significantly influences the mechanisms of action within the program theory. The program theory suggests a field of tension between two seemingly opposing phenomena, whose common occurrence represents the special feature of the intervention: the population addressed by Alles Clara is seen and not seen at the same time.

Modern, digital health technologies should be developed according to the principles of accessibility, transparency, privacy, confidentiality, and security [33]. The Alles Clara app as a tool for online counseling entails anonymity for the family caregivers as well as counselors. They can participate in the program without being physically seen. This also results in the flexible and low-threshold possibility of participating in the program for both sides. As health data protection should be a top priority of any digital tool [33], a strict data protection concept is an essential characteristic of the application. Furthermore, caregiving is still socially stigmatized and people who care for loved ones often do not self-identify as family caregivers [2]. In Alles Clara, caring relatives do not receive a label through their participation in the program and thus are able to take advantage of the service because they are not seen. Their inhibition threshold to register is low and they feel safe. They have the possibility to take advantage of counseling at any time, independent of time and place, quickly, easily, and “invisibly.” Since caregiving is time-consuming and resources are scarce, this is of great importance to family caregivers [7]. The counseling also takes place in a “distanced” manner. The caring relatives can write down their questions or challenges anonymously and get them off their chest. Even in the counseling room, the family caregivers are not seen by the counselor and can thus more easily enter a trusting relationship and address emotional issues. Shen et al. [41] describe the privacy perspective as a crucial mechanism to develop “eHealth trust” and to perceive benefit from the usage of health information technology. The phenomenon “not being seen” also characterizes the new work environment on part of the counselors. In Alles Clara, nurses work remotely from clients, which reduces physical stress and allows them to better balance their work with family obligations. Conditions that enable both a satisfying professional and private life make a significant contribution to a profession being attractive and in demand [4]. In addition, the work environment gives counselors more time to provide high-quality counseling. Job satisfaction is essentially related to being able to live the job-specific values in everyday work and to perform tasks in a high-quality manner [4].

Even if people in Alles Clara are “not seen,” the program is also characterized by the fact that they are recognized as individuals, as persons. They are “seen.” People as individuals are “at the center of digital health through the adoption and use of digital health technologies” [33]. The design of the app, the language, the color scheme, and the functionality are all aimed at making the family caregivers feel comfortable and addressed, and thus seen. Once they have registered, family caregivers have the opportunity to make their questions and challenges visible to the counselors in the digital counseling room. Both the establishment and maintenance of care for relatives is a dynamic process that is constantly changing and requires a wide range of skills from those who take on these tasks [2]. The counselors recognize the persons, their performance, and resources. In this way, the family caregivers can experience appreciation and no longer feel alone through Alles Clara. The counselors become visible in Alles Clara as well, with their knowledge and their competencies. By participating in Alles Clara, the nursing profession in particular benefits from being visible in a new professional context. In order to attract more people to a profession in nursing and care, the attractiveness of the areas of activity must be increased and, accompanying this, the image of care professions in society must be repositioned [4]. Alles Clara, as a new professional environment with a healthful work culture, in which the counselors recognize the quality and the meaningfulness in their actions, can lead not only to job satisfaction on an individual level but can also have a more far-reaching impact. The nursing profession becomes visible in and through Alles Clara as a mediator of knowledge and competence. Alles Clara can thus ultimately contribute to making the profession more attractive.

The overarching phenomena of “being seen” and “not being seen” are central to the impact of Alles Clara. While “not being seen” is an essential characteristic of digital services, aspects of “being seen” may be less obvious and expected. Looking past digitization, however, societies in Western industrialized countries are characterized by increasing individualization, which not only manifests itself in politics, public discourse, and lifestyles but also has an impact on health care. This increased attention to the personhood of patients has been described as a welcome side effect of such societal trends [42]. In times of increasing digitalization, the question arises as to whether digital tools can also live up to the claim of individualizing and humanizing health care [43]. The concepts of person-centredness and technology are often seen as incompatible, as closeness and warmth, which constitute person-centred care, would conflict with distant and cold technology [44, 45]. However, recent research demonstrated that innovative technologies could improve people’s relationships and sense of belonging [43, 46] and the program theory of Alles Clara suggests this as well. According to Jacobs et al. [46], it is the ways in which technology is used within health care that contributes to humanization or dehumanization. For the conscious use of technology-based support services, various aspects of person-centredness must be considered to prevent possible unintended dehumanizing effects of technology. This includes regularly inquiring about the needs, experiences, and perspectives of the users and giving them appropriate space. In this way, the development of technology-based services can be advanced while critically reflecting on and discussing ethical aspects, data protection, and personal rights [4]. Realist evaluation enables us to comply with this requirement. For this purpose, after the intended impact of Alles Clara has been theorized, a process and outcome evaluation guided by the preliminary understanding of the program theory will be conducted. We will adopt a multimethod, multiperspective study approach to investigate the initial CMOcs. While in-app tracking data and survey data will allow statements about the expression of the individual characteristics, qualitative interviews with both target groups, counselors, and people seeking advice, can help to uncover more latent mechanisms. This will allow the adaptation of the initial CMOcs and the refinement of the program theory.

5. Conclusions

Developing an initial program theory as part of the realist evaluation made it possible to develop a preliminary understanding that sheds light onto how Alles Clara unfolds its impact as a complex program. Contextual factors and mechanisms that condition these effects were revealed. This resulted in a configurational map depicting the complexity and logic of Alles Clara. The effect of the intervention unfolds on three levels: the app on the family caregivers, the counseling on family caregivers, and the new work environment on the counselors.

The initial program theory is not to be seen as a static or final model but is used as a basis for design, interpretation, and reflection of the following process and outcome evaluation. In this way, final results on effectiveness can be provided and the question can be clarified for whom and under which circumstances Alles Clara works and how. The results will provide information about the impact of Alles Clara for both target groups, show possible potential for improvement, and serve as a basis for regular monitoring.

Central to the discussion is that a digital counseling service such as Alles Clara shows its impact under the influence of the phenomena “being seen” and “not being seen.” In this way, the RE of Alles Clara can not only clarify the question: “What works for whom, under which circumstances and in which respect, and how?” but also offers a way to discover how technology can contribute to humanization, especially when it comes to the question of whether and how technical services can be supportive in the sense of a person-centred orientation of health care.

Data Availability

The data that support the findings of this study are available from the corresponding author upon request. The data are not publicly available due to privacy or ethical restrictions.

Additional Points

What Is Known about This Topic? (1) eHealth interventions can contribute to support informal care. (2) To date, it is poorly understood how app-based counseling and support services can contribute to relieving informal caregivers. What This Paper Adds? (1) Developing an initial program theory as part of a realist evaluation made it possible to develop a preliminary understanding of how the eHealth intervention unfolds its impact. (2) Findings suggest eHealth services can contribute to relieving family caregivers and benefit the professionals involved. (3) Realist evaluation offers a way to discover whether and how digital services can be supportive in the sense of a person-centred orientation of health care.

Disclosure

Theresa Clement and Katharina Gabl are co-first authors with equal contribution and importance.

Conflicts of Interest

The authors declare that they have no conflicts of interest with respect to the research, authorship, and/or publication of this article.

Authors’ Contributions

Theresa Clement and Katharina Gabl conceptualized the study, curated the data, performed formal analysis and investigation, developed the methodology, visualized the study, wrote the original draft, and reviewed and edited the manuscript. Jasmin Eppel-Meichlinger and Martin Wallner wrote, reviewed, and edited the manuscript. Hanna Mayer conceptualized the study, curated the data, performed the formal analysis, investigated the study, developed the methodology, wrote, reviewed, and edited the manucript, supervised the project, administered the project, and acquired the funding.

Acknowledgments

The evaluation of the Alles Clara app was financially supported by the Ministry of Social Affairs Austria. The authors acknowledge the support by Open Access Publishing Fund of Karl Landsteiner University of Health Sciences, Krems, Austria.