Abstract
Rich literature abounds concerning the clinical effectiveness of programs aiming to produce weight gain/obesity prevention outcomes. However, there is very little evidence on how these outcomes are produced, and what interplay of factors made those programs effective (or not) in the environment that produced those effects. This study aims to describe the application of realistic evaluation in the field of obesity prevention, as an approach to unravel those components that influence the capacity of a program to produce its effects and to examine its significance in an effort to understand those components. The concepts of critical realism have informed the development of an interview topic guide, while three European programs were selected as case studies after a rigorous selection process. In total, 26 in-depth semi-structured interviews were taken, paired with personal observation and secondary data research. Several grounded context-mechanisms-outcomes (CMO) configurations were described within the respective context of each location, with the mechanisms introduced from each project resulting in distinctive outcomes. This study highlights the potential of realistic evaluation as a comprehensive framework to explain in which contextual circumstances of each program’s effects are produced, how certain underlying mechanisms produce those effects, and how to explicitly connect the context and the acting mechanisms into distinct outcome patterns, which will ultimately form unique configuration sets for each of the analyzed projects.
1. Introduction
Critical realism is a philosophical position based on the work of Roy Bhaskar [1] which situates itself between the polarized traditions of positivism and relativism and stresses the need to explain reality as a product of science [2]. In their work, Pawson and Tilley [2] developed an evaluation paradigm of reality so that they could bridge philosophy with social explanations, and/or theory with applied research practice. According to Pawson [3]; “science is the business of understanding the unobservable structures and mechanisms that stand behind and produce concrete features of social reality” [3].
The critical realism assumption is that the current practice consists of an interplay between subjects and structures which pre-exist and which showcase certain properties and causal capacities. At each given time, humanity confronts a world consisting of earlier interactions of people as causal agents and structural conditions. Thus, in the forefront stands the fundamental question of the relationship between agency and structure, which within a critical realist viewpoint, possesses a realistic but also deep optimistic understanding of reality. Pawson and Tilley [2] establish the view that programs can be endorsed under favorable conditions, in supporting contexts, and enacted by capable agents. In the same way, programs can be disabled, blocked, and hindered by hostile structures and incapable agents. To make things more complicated, social phenomena are like biological phenomena, subject to emergence. Emergence was conceived as the distinction or the duality between parts and the whole, in both biological and social phenomena, as described by the Greek 4th century BC philosopher Aristotle in “Politics” [4]. In contemporary philosophy, emergence was again recognized for its importance in understanding complexity in “situations in which the conjunction of two or more features or aspects gives rise to new phenomena, which have properties irreducible to those of their constituents, even though the latter are necessary for their existence” [5]. Emergence is then to be understood as the “ever-changing” ingredient which adds to the complexity of social programs [6]. As such, the evaluation of social programs constitutes a non-linear explanation of the ever-changing contextual conditions, the ever-changing reasoning of involved agents and the mechanisms, eventually “ever-changing” as well, fired on by the social program under evaluation [7].
At this point, it would be interesting to draw the distinction between the social world as an example of an open system as well as a closed system. Starting from the latter, a closed system is one that appears to have controlled circumstances, and experiments of any kind can take place, which the scientist can then explain using successionist logic. One factor produces another, and the observer can predict or anticipate certain results by manipulating generative mechanisms according to the study preferences [8]. In contrast, the social world cannot be explained based on successionist logic because it is an open system, and the phenomena of the social world are influenced by a plethora of factors. Such factors can primarily be the geographical, historical, and social circumstances containing the rules, norms, and values of each era. This is the “context,” as conceptualized by Pawson and Tilley [2]. Other factors can be the so-called “generative mechanisms” which are the underlying drivers of the outcomes. The term “underlying” implies precisely the stratified nature of the generative mechanism, which is embedded in a stratified ontological reality. A generative mechanism can be the process of creating alliances and networks. The workings of the generative mechanism are not only unobservable but also cannot be recognized to function in a successive pattern of causation. Critical realism offers the option of explaining such open systems and understanding the complexity of their operation. The researcher should generate a theory combining human reasoning which is provided in response to the prevailing contextual ever-changing circumstances. An actor within a program decides and acts in a particular manner and responds to the opportunities and challenges of a context according to certain reasoning. People who come into a context with different resources have a restrained or privileged choice in the options and opportunities that are provided by this context. At the macro level, the collective reasoning of the agents, which comes as a choice within the opportunities provided by the social conditions, is what constitutes the outcome of a program [9].
That said, the interplay of context and generative mechanisms produces various outcomes, regularities, patterns, and/or associations, which is the primary subject of explanation in the form of understanding and producing a theory around “what works, in which circumstances and for whom” [2], under a realist evaluation perspective. Research will manage to discover and understand how programs work if the reasoning behind people’s choices within a context are understood; “measurement in social research will always be an act of translation” [3].
1.1. The Application of Realistic Evaluation in Obesity or Weight Gain Prevention
Obesity is on the rise in member countries of the World Health Organization, and for that reason, they appear to fall short of the targets set in 2011 by the United Nations High-Level Political Declaration on the Prevention and Control of Non-Communicable Diseases (NCDs) for no increase in adult obesity [10, 11]. Obesity constitutes a disease [12] and a major public health problem with wide personal, societal, and monetary implications. The strain is on the affected persons per se, the health systems, and society in general. The current COVID-19 pandemic has boldly highlighted the vulnerability of obese people not only to NCDs—as has been well documented—but also to communicable diseases [13, 14]. Calls to action have been intense in the last decades, but the COVID-19 pandemic has now created the urgent and undisputable need to both prevent and treat obesity.
The importance of obesity prevention with effective public health programs is the point of interest for this study. Evidence of what works in obesity prevention efforts is not self-explanatory, given the heterogeneity of the studies, the populations, and the settings [15]. Aside from this, when we refer to evidence on what works in prevention, we refer to what we deterred from not happening, the oxymoron of disease prevention: “when prevention succeeds, it creates an absence of events: indeed, when it works, prevention succeeds quietly, and it succeeds invisibly” [16]. That said, evaluation of health promotion interventions, such as obesity or weight gain prevention, promotion of healthy nutrition and healthy weight or increase in physical activity, has been the object of different frameworks for the evaluation of complex systems [17–19]. The fundamental starting point is the systematic medical evidence-based model which fails to view projects/programs/interventions as part of the reality of the social world. This reality includes actors and structures which operate at levels that produce events that exist, although they are not able to be seen or measured [20]. This study attempts to highlight the value of critical realism as a philosophical framework to view reality, and of realistic evaluation as the applied form of this philosophy in social research [21]. Thus, the realistic evaluation will be used as the methodology to explain and promote the understanding of weight-gain prevention programs operating and having effects on different levels of reality.
2. Methods
2.1. Evaluation Design
The research used a realistic evaluation approach, which draws on the philosophy of critical realism as a framework to understand those components that influence the potential of a program to produce an effect in the field of weight gain prevention. The programs included in the analysis were not chosen randomly. The selection of the three programs was based on a systematic literature review, which identified and initially screened 2,471 studies in the corresponding literature of interventions aiming to prevent obesity. A scoring system was developed based on a set of scientific and practical criteria. The scientific criteria included a scoring system based on the “Obesity Systems Influence Diagram”—described by the Foresight Report [22]—to rank the results of the literature review according to the number of thematic clusters that each program addresses, the intervention duration of no less than 24 weeks, the sustainability of the effect (s), and ultimately if the existence of measurable outcomes was based on the program evaluation. The existence of measurable effectiveness in delivering the intended outcome results from an evaluation process of measuring the effectiveness of the program; otherwise, the program could not be considered, because the construction of context-mechanisms-outcomes (CMO) configurations would not be possible with no measurable outcomes.
That said, after applying these criteria, the candidate studies were reduced to 11 studies. Finally, a set of practical factors is considered, such as the feasibility to find the team which fulfilled the research, if the author could speak the language of the setting in which the program took place, and if the team would agree to participate in the inquiry, among others. Three programs were finally selected as case studies, which were considered “effective,” “successful” and/or “good paradigms” of interventions having produced measurable outcomes. Although the description of the scoring process is not part of the present article, its details are thoroughly described [23]. The sustainability factor proved to be a key factor because two of the three programs, although “old,” are operating successfully till today, and have been transformed and adapted accordingly. It is also believed that the outcomes of the program are still meaningful in today’s world. The Healthy Weight Communities project is not active nowadays, but its effects as a pathfinder community are well demonstrated and integrated into future public health practices [24].
Each case study was set up as an independent qualitative inquiry with semi-structured in-depth interviews, combined with personal observation research, in two of the three case study settings, and with secondary data research on the settings. The combination of all would constitute a realistic synthesis, to explore the arrangements of the mechanisms, contexts, and outcomes acting behind each program.
The conceptualization of this research was devised after the common need to recognize what happens when a program changes setting or time or participants. Is there something we can understand about the way a program fires its resources in each context, which produces a reasoning that “works” further beyond the particular setting and can have a transferable value for our knowledge? The illumination of the reasoning as the microelements that a project has introduced by its operation in a particular setting was sought to be identified using a realistic evaluation approach in the present study.
2.2. Setting
Three programs were selected as case studies (based on a systematic application of a set of criteria): The “Healthy Weight Communities” (HWC) project which took place in various cities of Scotland, the “Movement as Investment for Health” (BIG) project in the city of Erlangen, Germany, and the “Walking for Health” (WfH) project which took place throughout the whole of England.
In addition, the Paideiatrofi program in various cities of the Athens region served as a pilot study program. The project was based on the EPODE (Ensemble Prévenons l’Obésité Des Enfants: Together Let’s Prevent Childhood Obesity) methodology [25], which was implemented in France for more than a decade. Paideiatrofi in Greece was based exactly on this methodology but was initiated during the period that the current research project was launched; thus, its evaluation results were not yet published. The existence of measurable effectiveness in the form of an evaluation process of measuring the effectiveness of the program, as explained previously, was deemed to be a necessary criterion in order for a program to be considered as a candidate study.
2.3. Participants
Participants in the interviews included the employees, key stakeholders, and coordinators involved in the initiation, organization, operation, and administration of the above-mentioned projects. The potential interviewees were identified through scientific congresses that participated, presenting the conclusions of the projects, and the initially identified persons were used as reference persons to create further contact cycles among their colleagues. From the initially identified persons, none declined to provide an interview. A written informed consent form was signed by the interviewees before the interview process, informing people about the purpose of the study, the content of the interview guide and a protocol of the measures taken to ensure the confidentiality and anonymity of their identity. For a specific interviewee where it was quite possible to be recognized from their prominent role in the project, a consent form specifically written for the purpose was foreseen, requiring consent under the option of possible identification.
Ethical approval was not deemed necessary for the execution of the study, since the topic guide of the interviews included the provision of information on the organizational operation of the project, which was not personal, private, sensitive or confidential in nature.
In total, 26 in-depth semi-structured interviews with a duration of 50–80 minutes were taken from the participants within the organization of the three projects that were initially selected as case studies (HWC: 12 interviews, BIG: 6 interviews, WfH: 8 interviews). Each participant was assigned an identity code. Another three additional interviews were conducted in Athens in the framework of the Paideiatrofi program, which as explained was not included in the final inquiry but served as a pilot project to test the validity and the comprehensiveness of the topic guide. Those interviews were translated into English but were not analyzed.
2.4. Data Collection
For reasons of clarity, the process that was followed for the research is described in five discrete steps in Table 1. Steps 1–3 describe the data collection process and Steps 4–5 the data analysis process.
The next section presents the results of the three case-study projects which took place in Scotland, Germany, and England, and included a measurable target to promote healthy weight levels and prevent obesity, although their aim was not headlined per se as such; neither were any of the three programs branded in this way. A brief overview of the content and the context of each project is followed by an analysis of the most important context-mechanisms-outcomes configurations. The tables at the end of each project’s analysis aim to highlight the abstract descriptions of the CMO configurations as somehow unconnected from the personified characteristics of each case study. Thus, they offer the structure of the CMOs in the form of a storyline without any hint of identification of the project’s features. This type of description aims to deliver the essence without the details of each existing project. That said, the Healthy Weight Communities (HWC) project is headlined in the table as a health promotion project, Movement as Investment in Health (BIG) as a participatory project, and the Walking for Health (WfH) project as a walking project.
3. Results
3.1. HWC (Healthy Weight Communities)
The first case study was the Healthy Weight Communities (HWC) program that took place from 2009 to 2012 in seven pathfinder communities, namely Viewpark in North Lanarkshire, Dundee, Dumfries, Priesthill and Househillwood in Glasgow, Armadale and Blackridge in West Lothian, Catrine in East Ayrshire, Barrhead in East Renfrewshire and Stevenston in North Ayrshire. The project was about bringing a pool of activities related to healthy nutrition, healthy weight, and promotion of physical movement to communities, and especially young families. The HWC project—similarly to the Greek Paideiatrofi project—was based on the EPODE (Ensemble Prévenons l’Obésité Des Enfants: Together Let’s Prevent Childhood Obesity) methodology [25].
Eleven interviews were taken in the period from 7 April 2011 till 18 November 2011, with one face-to-face and ten telephone interviews, since it was not feasible to arrange interviews within a short time framework; thus, they were scattered throughout a 7-month period. Participant observation was not an option in this project, as was the case with the other two projects. For this reason, a larger number of interviews was conducted to compensate for the lack of participant observation during the collection of the data. Moreover, demographic information on the communities and photo archives from the project events were requested, in order to visualize the character of the project.
3.1.1. Context
The depth of sophistication that public health viewed within the communities reflects the existence of an ethos and a culture of thinking about public health with structures and budgets which are recognized for their importance. The long-standing accumulation of projects in the communities with the intention to bring social change has brought the level of sophistication and advancement in public health delivery to a high level among European countries. This fact was reflected in the public health language used by the interviewees and it was also a product of the description of the public health services available in the communities. There was an institutional memory that needed to be further maintained and promoted in the form of partnership working and capacity building. (6) Thus, the task of the HWC project was to orchestrate and bring together in meaningful partnerships discrete activities that were regarded as unconnected under the umbrella of healthy weight promotion. This was often acknowledged in the interviews and was set centrally in the objectives of the project with clarity:
[I have always felt that we needed to …bring in the services together, actually deliver more than one message… we can no longer just target one activity… We need to bring them together, so for healthy weight, we need to start working across services and across partnership activities …we need to have a sports development team giving the message about nutrition...] [23].
3.1.2. CMO 1
Applied upon an overall context of a partnership-working culture within areas that were regarded as interconnected activities, the HWC project created the need for a new model of work in the so-called pathfinder communities. These pathfinder communities had the mandate to open new “paths,” new cooperation models which would incorporate weight management in their daily practice, driven by the conclusion that their existing working patterns were not adequate to handle the weight problem that was observed in their communities. Thus, a different model of working acted as an enabling mechanism in the context of an existing partnership and collaborative culture to produce new perspectives of working enriched by practitioners stemming from different backgrounds. The new model of work had the weight management issue at its core and how it can transform the objectives of different local authorities, ranging from Leisure Services to Neighborhood Renewal Units. In the framework of the project, the main objective of the Neighborhood Renewal Units remained the improvement of the local infrastructure’s level, but it was also setting a new dimension in terms of how this improvement would contribute to helping users become more physically active. The new model of working required the empowerment and the enabling of the partners to understand how discrete council authorities would come together under a common purpose. For this reason, education and literacy activities were required to promote capacity building among officers to support social change and understand the complexities. Capacity building, according to the interviewees, was improved by educational and training programs (e.g. social marketing principles, use of social media, social change research, and health promotion), support for individual Continuing Professional Development, skills development for strategic and partnership collaboration, access to the use of evidence, peer networks and collaborative sharing of knowledge, mentoring and coaching, and both technical and learning support. Thus, by using the vehicle of healthy weight promotion, the HWC project inspired a change from partnership working within services to partnership working and interconnecting along diverse services within the council.
3.1.3. CMO 2
The different mentalities and cultures of people from diverse councils and boards constrained the potential of the program and it was not a given that the mandate of the project would work automatically. The description of the work could not become modified or integrated with other agendas overnight. This compromised the ability to proceed with the agenda of the pathfinder communities. Another structural feature was the timeframe set by the project which was deemed inadequate to promote changes among the recipients of the community services as well as the community services themselves. Time was often referred to as the factor which would allow maturity and cultural shift in the behavior.
3.1.4. CMO 3
In an environment where residents of the involved community are frequent users of council services, implying that reliance and confidence on their usage is high, the project’s potential was expanded by a mix of enabling mechanisms. By having a pathfinder character, the project had the autonomy to heed the needs of each community and make adjustments according to those needs, appearing to the project manager as a priority that must be fulfilled. Thus, being flexible and adaptive were properties that emerged from both the structure of the project as such and the operational adjustments made by the practitioners according to the needs of each community. Among the project’s cognitive resources was also the determination and enthusiasm to increase the outreach and diffusion of their messages to those that were most in need of adhering to that healthy weight messaging which was being “fired out” by HWC. Working along a social marketing strategy pathway, the project demonstrated a vibrant and positive profile, with emphasis on the entertainment factor, and consciously adopted a neutral and non-stigmatizing vocabulary towards body weight.
Thus, social marketing and capacity building are key factors in achieving community engagement. Another enabling factor was to allow enough time for communication work to grow in the community and let people gradually familiarize themselves with messages about healthy weight levels and the value of healthy cooking, and to increase the usage of free spaces, and create opportunities for physical activity. The key factor for increasing physical activity was the vibrant and non-stigmatizing marketing strategy, which highlighted the “fun” factor, and committed the project to become mainstream by allowing for time and consistently creating opportunities for people to familiarize themselves with and adhere to the new messages.
Being able to focus on the needs of the community enhanced the ability of the project to produce results by increasing the usage of green space for physical activity, healthier cooking techniques and recipes, and adherence to healthy habits promoting weight optimization. The increase in usage among people with overweight issues, as measured by the evaluation results of the project, implies that the project messaging offered the necessary resources for people to overcome their individual barriers of weight stigmatization, thereby seeking avenues to make changes.
Another main outcome was the social cohesion as promoted by the increased local participation in the projects’ events, and first of all, its ability to attract children using a modern social marketing strategy, and then through this channel to attract the whole family as well. A summary of all the configurations of the first case study can be seen in Table 2.
3.2. BIG—Bewegung Als Investition in Gesundheit (Movement as Investment in Health)
The second case study was the BIG project. BIG is the German acronym for “Bewegung als Investition in Gesundheit” (in English: Movement as Investment in Health), which had the aim to promote physical activity among women in difficult life situations, such as migration, poverty due to unemployment or other adverse family circumstances. Following the health promotion principle of the World Health Organization, the project not only targeted their fitness status by offering physical activity classes, but it also enabled the participants to actively be gaining control of their overall health and the determinants that help them stay healthy. A comprehensive participatory approach was applied and women were included in planning, implementing, and evaluating the project [26]. Following a cooperative planning process with local authorities, the project featured representatives of the target group, offering them low participation-fee exercise classes, including childcare, women-only swimming pool hours and classes, and involvement of the target group of women in the further organization and dissemination of the offerings in the community [27]. The implementation phase of the project was part of an Erlangen University-based scientific project, which gradually, over a period of 3 years, became a community focal point, with a mandate to enable access to the corresponding target group to sports facilities [28].
Methods for the analysis included participant observation in the physical activity courses, document analysis, and qualitative interviews with six people involved in the project’s administration, which were conducted face-to-face from 30 November till 2 December 2011 at the premises of Erlangen-Nuremberg University and the Municipality of Erlangen in Germany.
3.2.1. Context
Erlangen, with over 110,000 inhabitants, is a city in Bavaria ranking high in the socioeconomic index as measured with the power purchasing parity index of German cities of the corresponding year when the interviews were conducted. Being a city that hosts the headquarters of major German and international firms, it attracts professionals with relatively higher educational status than the average German city. Erlangen is known as Siemens-city; together with Nuremberg-Erlangen University, they are the biggest employers in the area. Its population consists of diverse ethnicities, made up by almost one-third of non-Germans, following a background that was also stamped by the integration of the French Huguenot refugees from the 17th century. At the time of the interviews in 2011, the authorities were placing the provision of services to the less well-off and migrants as a priority on the political agenda. This was driven particularly by an administration which at that time brought together a steering committee for integration, with an emphasis on understanding migrants’ needs and incorporating intercultural thinking into the provision of services. The notion of equity and intercultural integration still dominates the process of decision-making in Erlangen.
3.2.2. CMO 1
In this wider context of encouragement for such initiatives in the city of Erlangen, a key role in the design, acceptance, and implementation of such a program was played by the Project Director, a figure with expertise in the field and the personal capital to overcome structural obstacles. Thus, as analyzed above, in such fertile ground, the University of Erlangen dropped the seed for a project for physical activity/integration under the steering of a professor with wide public acceptance and recognition from the local authorities. The project used the approach of “enabling” policymakers and professionals to produce action and create social change in the local, political and institutional setting [29]. However, the strength of the Director’s presence as a facilitating mechanism to initiate action as applied on the ground, in a community that is prolific for integration projects, showed to be the appropriate arrangement that helped to produce the “birth” of the project’s approach.
3.2.3. CMO 2
The approach of the program was based on creating within the organizational level of the project an empowering and enabling milieu for the participating women. The identification of women with extended contact networks and their usage as “social catalysts” played a twofold role in the proliferation of the project. On the one hand, through these women, the project could increase its outreach to the women who were in most need, and multiply its penetration to the target group; on the other hand, the women themselves could develop feelings of self-trust and self-belief. The fact that they participated along with local authorities in the process of designing physical activities “with” them and “for” them was empowering and inspiring in the sphere of the intangible outcomes of this process:
[…they felt they were taken seriously, yes? This was the core of our approach. They sit around the same table with the mayor, with the decision makers and they are fully equal partners…] […this is why we plan physical activity courses with them for three hours where three experts would sit together and in ten minutes they would be ready.] [23].
On a practical level, the cooperative planning process had as an outcome a range of activities that were fulfilling the needs and schedules of the target group in a practical format. This format included the arrangement of babysitting services during exercise sessions, usage of neighborhood school facilities, low fees, and women-only hours in the local pool for those whose religious beliefs constrained them from being with men. Moreover, the issue of personal negotiation or family arrangements before committing to certain hours and days was very important for the effective planning of the sessions. This type of approach, also discussed in Deem and Gilroy [30]; showcases the role of structural conditions in facilitating or constraining participation in sports and the conditions of negotiation with partners as a strategy to encourage adherence to physical activity. When sports educators acknowledge the need for negotiation with the familiar network, there are more chances to formulate sustainable forms of physical activity that promote lifelong learning. Thus, it appears that the factors which increase physical activity are empowerment and a climate of trust and understanding of the personal conditions. Creation of win-win situations and negotiation of conditions that work appeared to promote physical activity with sustainable anchoring on different aspects of the target group’s lives.
As a result, the same women who felt empowered emerged to act as social catalysts and attract further women from their networks. At first, with continuous private efforts, the project team identified within their life circumstances women with extrovert profiles, who could take part in the cooperative planning sessions. Then, those same women acted as social catalysts to identify further women within marriage ceremonies, school gatherings, and other cultural occasions, and get them to join the physical activity groups. Once the women realized that they liked the idea of joining these groups, they felt motivated to invite other women from their private networks, and so the cycle could continue.
3.2.4. CMO 3
In a wider context of a relatively prosperous city possessing local structures that had the capacity to allocate resources to health-promoting activities, some factors simultaneously had hindering action on the implementation of the project. The dilemma of allocating public resources to one target group and not to another comes as a process of natural competition within policy-making, and it was not always self-explanatory for all stakeholders. This “dilemma” takes the form of opposing voices who have different opinions, developing resistance with arguments that put under scrutiny the prioritization of one target group or another in the allocation of scarce public resources for the purposes of health promotion. Often, alternative groups were proposed, such as health promotion among children instead. On the other hand, there exist cultural conditions which are reproduced over the years, monopolizing the order of things and creating silos of power, often with one-dimensional views. Such disabling mechanisms formed the generalized culture of providing access to sports facilities only to sport clubs; thus, the request of the project’s organizers to use swimming pools for women from diverse backgrounds appeared to be culturally unfitting. Such difficulties were sometimes either not resolved and the project team chose other ways to proceed, or it allowed for appropriate time and space to change attitudes and find a future opportunity which could be more convenient. Sometimes, changes in the political landscape created the political windows that were necessary to proceed with the agenda of the project, with new persons who remained positive about the project. New persons in key positions might be more helpful in the agenda of the project, acting as enabling and supporting factors.
The above example is indicative of the project’s effort to provoke a structural fracture in the existing conditions, where possible–because sometimes it was impossible—to enter the status quo, and become part of the new status quo with emergent properties. This effort was sometimes in conflict with the academic character of the project, which required, for instance, implementing an evaluation schedule in the form of a randomized controlled trial with one group receiving the intervention and the other group serving as a control and not receiving the intervention. The organizers decided not to compromise the project’s impact on the community, and create a group of women, who did not receive the intervention so they decided against that form of evaluation. Other compromises were found in the reduced frequency of visiting the exercise groups, where the project team allowed for time to make adjustments, and showed respect for the cultural and personal circumstances of the target group.
3.2.5. CMO 4
In contrast to the following project which has non-health-related motivations, such as socializing with other participants to increase participation in physical activity, the BIG project uses physical activity as the vehicle to empower the target group and increase participants’ integration in society, with active participation in life activities. Exercise is used as a vehicle to promote their self-realization and self-effectiveness, given that the target group are women that find themselves in difficult life circumstances. Finally, physical exercise is used as a camouflage for effective integration in difficult-to-reach groups:
[from the beginning, we were pretty sure that this is not only about sports but about integration through sports too] [23].
A disabling factor for this integration function was the formation of cliques among women who knew each other. This atmosphere was found to be unhelpful in terms of the integration of new women who were entering the exercise groups. The catalyst in such cases was the personality of the instructor to create a welcoming climate for new members.
The project was successfully transferred to the community in Erlangen when the academic project’s duration and funding came to an end. The factors that enabled this transition were the training and qualification of women as sports assistants for culturally diverse groups. Moreover, an important role was played by the development of the documentation of all the implementation steps of the project (the BIG Manual) which facilitated the knowledge transfer from the university to the community. Overall, this transition was enabled by inserting from the beginning of the operation the necessary regulations, and proactively facing the potential problems that would hinder the planting of the project into the community and its sustainable operation in the following years. A summary of all the configurations of the second case study can be seen in Table 3.
3.3. WfH (Walking for Health)
The Walking for Health (WfH) project is a support network of walking schemes across England. The phrase “walking schemes” refers to the coordination of health walks, which are defined as “purposeful brisk walks undertaken on a regular basis” [31].
Eight interviews were conducted face-to-face in November (1–28) 2011, in the following cities of England: Nottingham, Worcester, and Peterborough.
3.3.1. Context
Walking as an activity has a long tradition in England, either as a solo activity or with organized group schemes on a regular basis. This long tradition of walking is to be understood as both a sum of institutional networks, which maintain a widespread ethos of walking as a socially acceptable activity, fueled by the reasoning that participants give when engaging in walking activities. The Walking for Health program belongs in this context, as the idea of a General Practitioner who started the WfH initiative back in 2000 within the Countryside Agency, organizing easy walking schemes for free to the public all over England based on volunteer trained instructors. In 2006, WfH became part of Natural England and acquired a clear organizational structure with a paid national management team which was funded since 2009 by the Department of Health. By 2011, WfH encompassed more than 500 relatively short walks on easy terrain, aimed at those with low levels of physical activity. By the time the interviews took place, WfH was under transition with a new funder who was unknown, and relative uncertainty was evident. Other long-standing walking schemes, such as the Ramblers, were promoting brisk walking and the protection of the physical environment; they normally encompassed longer walks and covered more difficult terrain. Nevertheless, nowadays, those two walking schemes are cooperating and share a common vision [31].
3.3.2. CMO 1
An extraordinary form of belief was clearly discerned very early in the analysis of the interviews. Different aspects of this feeling ranged from dedication to the idea, faith in the aims of the project, and personal gratification from its positive impact on other people. The initial hypothesis that belief in the project was a core mechanism of the project’s function was soon established, and a further investigation aimed to illuminate the source of that belief, which is what fueled it and what helped sustain it throughout the duration of the project.
In relation to the origin of this belief in the project, there were different “small” theories to explain it. The first theory is the interplay with the existing background on walking and the emergence of new reasoning for the participants. As stated above, the project was operating within a context of a strong historical tradition of walking. People were familiar with walking as a core activity and possibly a thread of companionship, along with discovering new places in nature, preserving nature, traveling long distances, being adventurous, and the solitude feature of the walking activity. Walking for health offered new reasoning and a new sequence of understanding; as such, it was quickly received and adopted by people who wished to walk but were afraid to join more vigorous schemes. Thus, the appearance of a project with “smaller” proportions and a modified intention for the same purpose shaped new reasoning among the potential participants. As a result, it could appeal to people who were willing to do an easy walk within a “become-more-active” scheme, but were not willing to join a disciplined walking scheme with environment-related purposes, longer duration, and a more difficult trail. With the increase of participation in healthy walks, the gain of considerable benefits was reported, which led to collective positive feelings with regard to the operation of the schemes. Modest expectations from physical activity-oriented programs appear to be more open in scope and address a wider audience, and are thus more effective in achieving an increase in bodily movement and progressively becoming moderate and vigorous physical activity.
Concerning the role of the General Practitioner who pioneered the idea of the walking schemes, similarities with the previously discussed BIG project and the influential role of the Project Director could also be noted. As Pawson and Tilley [2] advocate, “differently resourced subjects make constrained choices amongst the opportunities provided” [2]. The fact that the project was based on the action of a person could be a source of inspiration and faith to continue this path.
Furthermore, its freedom aspect and the capacity of an individual person to initiate action in an environment, to resolve tensions, and to hinder structures that were not in line with the vision of the action are some of the initial stages in a chronological pattern of events. If we could locate the projects on an axis of structure and agency at their ends, the BIG project was closer, in chronological terms, to the influence of agency. On the other hand, when the interviews were contacted for the WfH project, the observer could see an institutionalized project, or realistically speaking, embedded in the structural conditions. Thus, chronologically speaking, the project had passed from the stage of a person or different persons influencing its trajectory, to becoming a part of the organizational and collective order of the structure. If the previous BIG project was granted a sustainable future and could become rooted in the existing and ever-changing structural framework, then the impact of the initiator as a person would some years later become like WfH, using the past to tell the story of the origins of the project.
3.3.3. CMO 2
The analysis showcased how the positive feelings soon became the mainstream climate, inviting more and more people to participate in the walking schemes. The creation of this atmosphere became the flagship achievement of the organisers and a source of personal gratification as well as forming the purpose of their job for its social benefit. The dependence of the project on creating feelings of gratification and commitment to serving other humans was a critical characteristic of its operation, since the walking scheme leaders were participating voluntarily. There was no financial reward and no payment fees from scheme participants. They were receiving training from the project’s institutional structure; then, they had the mandate to guide the walking schemes voluntarily. Receiving feedback about the rewards and benefits of different aspects of the participants’ health contributed to reinforcing—and more importantly maintaining—their sense of commitment to their voluntary mission. The outcome was, on the one hand, a feeling of gratification from both the employees and the volunteers of the project and the sense that they were fulfilling a mission; on the other hand, the recipients were increasingly in search of new reasoning that soon became obvious: they were receiving intangible benefits in the form of better mental health through socializing within the schemes, finding friends, and overcoming solitude. It was reported by the organisers (WfH4, line 873–875; [23], p.208) that many participants in the walking schemes could regain a social interest and purpose in their lives after the loss of a spouse or a serious illness. The benefit of increasing their physical activity was acting on a second layer, becoming an indirect outcome and a by-effect of walkers’ participation in the schemes.
Another source of belief stems from the type of operational environment that most employees perceived to be working in, the level of management principles, and the quality of skills they were acquiring, with first and foremost the opportunity to improve their interpersonal and communication skills. This type of evidence was evaluated as valid because it stemmed from a mix of interviewees with a professional background that was not homogenous, ranging from the public sector to international corporations. Moreover, some interviewees’ contracts were soon to be definitively terminated due to the transition of the project to another fund-giver, which was thus deemed likely to dilute the personal benefit, given that the occupational environment level was of a higher caliber.
Another source of belief in the project is related to the nature and core idea of the project itself. There was a widespread belief that the project was about a straightforward concept, which easily attracted participants, given that it was implemented and operated structurally. WfH had a clear organizational structure through a national management team with an enabling function. This included having the know-how to enable capacity-building towards forming walking schemes around England. Paired with the stimulating working content in terms of acquiring new skills and expertise, the flexible operating hours and the work-life balance, it reinforced the idea that participants enjoyed being employed in this project, raising the team ethos and acting as a reinforcing reasoning to keep their faith in the project.
3.3.4. CMO 3
Another mechanism of action that the data analysis illuminated was the aim of the project to follow a win-win strategy to promote the establishment of new schemes in the communities all over England. The dissemination of the project was possible with an “after-sales package” which included a free training bundle for walk leaders, insurance for the walkers and access to the technical resources of the project. A similar approach was also followed to promote the establishment of partnerships, with possible stakeholders such as local supermarkets that advertised the walking schemes, also offering small discounts for beverages served at meeting points, with the benefit of increased product sales. The walking schemes were a meaningful target for the primary care trusts (PCT) which could partner with and take advantage of the structured expertise and functionality that they could incorporate with minimum expenditure. The creation of win-win arrangements not only promoted the establishment of walking schemes but also increased its sustainable operation and the engagement of the stakeholders.
3.3.5. CMO 4
The mechanism of creating enjoyable and sociably pleasant experiences appears to be a key factor in increasing physical activity. The project employed a marketing strategy that emphasized the pleasure factor and the enjoyment aspect of the activity as the main motivational factor to increase the number of walkers and maintain their involvement in the schemes. Fun and joy as priority factors create a non-health-related motivation as the central concept of a project, something which is recognized in the literature, such as in Robertson et al [32]; whose study included 7,307 subjects from the Health Information National Trends Survey 4, highlighting “enjoyment” as the factor that had the highest effectiveness in increasing adherence to physical activity programs. Thurston and Green [33] also recognized the need to conceptualize meaning to involve the holistic creation of relationships and self-development in the field of exercise prescription as essential ingredients in promoting participation and adherence to exercise. A summary of all the configurations of the third case study can be seen in Table 4.
4. Discussion
Traditional evaluation can only provide a snapshot view of the mechanisms that produce a change in the operation of a system. The idea of the intervention was like a black box that should be illuminated as a suitable concept. This type of realistic inquiry can safely claim that it manages to capture a piece of wisdom that usually gets lost within the history of an intervention and is rarely apprehended in its evaluation:(1)The subtle arrangements that were made in response to the challenging situations in both a conscious or unconscious manner: Although one may wonder how these are to be defined and, with even more difficulty, to be expressed, if they are unconscious, there is an acknowledgment that the sum of the experiences, handling, and interactions of the involved practitioners during the project will become a distinct act of direction in the order of the events. In other words, how operators reacted to the disabling situations emerging during the project and what corrective settlements they put into the course of the events to arrange the resources of their program adequately to fit the circumstances is an overall dynamic situation that will be lost as a clue to its success, when traditional evaluation only asks “how much?” and not “how did this program work to produce its effects?”(2)The common wisdom applied by the practitioners, which is usually not included in the traditional evaluation of effects: People have lived through experiences, they carry an institutional memory, they have recollections and store an intangible collective reservoir of information, which goes uncaptured without an effective evaluation. This personal storage becomes collective and constitutes working wisdom that comes naturally with daily interaction within a context and its resources.
The application of realistic evaluation in the three case studies in the field of obesity and weight gain prevention in general was a complex task. The analysis of diverse social conditions in three different countries (Scotland, Germany, and England) was alone a challenging task. The case studies were also diverse in their scope, as well as in their target group.
Nevertheless, applying realistic evaluation had the effect of bringing those diverse case studies together in terms of understanding how health promotion works. Social circumstances, combined with the inner workings of a project, enable or disable outcomes. Usually, those effects or results are to be reported in epidemiological papers in the form of quantified sums. This is a valuable document to have from a project running in a community or a city; it is desirable for the public using the program and/or the funders to get a measurable depiction of the effectiveness of a program on various parameters, for example by increasing hours in the physical activity of a target group or increasing the consumption of vegetables as measured in units. Despite the scientific need to minimize and reduce the effects of a program on a unit of effect, there is an imperative scientific need to explain “what works, in which circumstances and for whom” [2]. This need probably refers to another level of reality because it entails all the unconscious reasoning of people to make a program work or not work. The underlying generative mechanisms that produce the effects of a program in health promotion inform policy in relation to how outcomes are produced.
Programs are open systems with fluid conditions and what works in a time-space location may not be effective in another one [34]. Archer [35] warned of the “morphogenetic nature of society as its defining feature, the capacity to change shape or form” [35]. So, what is the purpose of research on systems that are time- and space-specific? The purpose is to extract, through the method of replication and repetition in diverse settings, such as the present study, those truths that resonate among a group of people and apply universally.
The aim of this research was to illustrate the value of realistic evaluation at illuminating the inner workings of three health promotion programs and to establish the application of realistic evaluation in the field of obesity prevention. Further research is needed in terms of fleshing out those universal truths that apply across the three case studies, which can provide a more realistic theory that can then be informed with further specifically-set cycles of the context-mechanism-outcome (CMO) configurations [2] for the purposes of policy-making and best practices.
Thus, the number of interviews conducted in each case study was enough to reach an adequate level of understanding of the most important components of each program and establish the CMO configurations that can explain the operation of each project. An acceptable level of saturation was achieved but a further cycle of interviews might have helped to better refine those configurations further. However, it must be noted that the scope of this study was not to provide detailed accounts of all the mechanisms on display in each project, but indeed to showcase the significance of a realistic approach in understanding “what works, in which circumstances and for whom” [2].
The potential of critical realism is in line with the recent conclusions from Quintans et al [36] who, in their review of critical realism, described a guide of 21 stages that help the researcher to implement an evaluation of critical realism. This guide is informed by an integrative review of methodological and practical realistic evaluations, and it is regarded as overlapping substantially with the methodology followed in the current analysis of the three case studies on obesity and/or weight gain prevention.
Data Availability
Electronic-form transcripts of the conducted interviews and signed consent forms are all in possession of the author. The nVIVO software, which was used for the analysis is not available anymore because the license was not renewed. The thesis can be freely accessed at the following link: https://wrap.warwick.ac.uk/74065/.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The study was a Doctorate Work in Warwick University and was funded solely by own means. This submission was funded by the Library’s Publications Fund at Hanover University of Applied Sciences and Arts.