Impact in Participatory Health ResearchView this Special Issue
Review Article | Open Access
Erica Di Ruggiero, Nancy Edwards, "The Interplay between Participatory Health Research and Implementation Research: Canadian Research Funding Perspectives", BioMed Research International, vol. 2018, Article ID 1519402, 7 pages, 2018. https://doi.org/10.1155/2018/1519402
The Interplay between Participatory Health Research and Implementation Research: Canadian Research Funding Perspectives
Objectives. The objective of this paper is to investigate what participatory health research (PHR) can offer implementation research (IR) and vice versa and discuss what health research funders can do to foster the intersection of both fields. Methods. We contrast points of divergence and convergence between IR and PHR. We reflect on whether community engagement is necessary and on the unintended consequences of participation in IR. We describe how a research funder can incentivize PHR in IR. Results. Participation is encouraged in IR but the nuances of who is involved merit greater attention in IR. PHR and IR differ in emphasis placed on the scale-up of the intervention. However, they share a common interest in generating real-world contextually relevant evidence. Conclusions. We need to assess whether and how funding practices influence researchers in how they undertake PHR. Researchers need to better account for participatory approaches to ensure that any potentially harmful consequences are described (and better understood so they can be mitigated in the future) and elucidate the ways in which these processes do or do not enable implementation and scale-up of interventions in IR.
Increasingly, researchers and research funders around the globe are turning their attention to funding and conducting implementation research (or science). This growing trend is in response to a need for real-world and contextually sensitive evidence to respond to and solve implementation problems facing policy-makers, practitioners, communities, and other social actors working in sectors such as health [1, 2]. Lack of community readiness or engagement and a poor understanding of the sociopolitical and cultural contexts in which interventions are implemented are among the reasons cited for this limited progress in using evidence in policy and practice settings [3, 4].
“Implementation science examines what works, for whom, under what contextual circumstances, and whether interventions are scalable in equitable ways. The intervention includes the “what”—[for instance, a policy or community program]—as well as the “how”—the implementation activities that are required to achieve full (equitable) coverage of …the intervention” . Like with any emerging field, a plurality of research methods and theories are expected to contribute to the further conceptualization, design, and conduct of implementation research studies. In recent decades, interest in participatory health research (PHR) has been steadily rising, in part due to increased funding incentives for academics to cocreate knowledge and align research with the real-world needs and interests of different actors with the promise of greater applicability and impact of the research for communities, policy-makers, and practitioners [5–7].
In light of such trends, research funders are responding. The Canadian Institutes of Health Research (CIHR), Canada’s premier health research funding agency, supports research that is either investigator-led or priority-driven across the four health research pillars/themes (biomedical, clinical, and health services and social, cultural, environmental, and population health) . While PHR is generally applicable to three of the four pillars of health research at CIHR, this article focuses on research funding initiatives addressing two of the four (health services and social, cultural, environmental, and population health). CIHR has, over the last five years, stepped up its contributions to implementation research (IR) through its investments in global health research (e.g., maternal and child health, chronic diseases) , Pathways to Health Equity for Aboriginal Peoples , HIV/AIDs , and various health systems and patient-oriented research initiatives to name but a few . CIHR has also declared its commitment to citizen engagement in health research, which it defines as the meaningful involvement of individual citizens in activities such as research priority-setting, planning, governance, and peer review to enhance the relevance and application of research into practice and policy .
In this commentary-style article, we pose the following central questions: what does participatory health research (PHR) have to offer implementation research (IR) and what does IR have to offer PHR? We start by looking at how PHR and IR diverge and how they intersect or converge with respect to participatory approaches. Then, we briefly consider the following questions in relation to implementation research: Who are the “participants” in IR studies and is community engagement always necessary in IR? What are the possible unintended consequences of participatory processes in the context of IR studies? Using illustrative examples from recently funded research initiatives, we also discuss what roles health research funders can play in fostering the intersection of participatory health research and implementation research and consider how this would advance research in both fields.
3.1. What Does Participatory Health Research (PHR) Have to Offer Implementation Research (IR), and What Does IR Have to Offer PHR? How Do They Intersect and Diverge from Each Other?
While it is well beyond the scope of this paper to review all possible theoretical and methodological approaches that underpin PHR (refer to other papers in this supplement), suffice it to say that PHR approaches hold some common core principles. These include meaningful community engagement and cocreation of knowledge that accounts for different ways of knowing, commitment to social change extending beyond the generation of knowledge, grounding in participants’ lived experiences in the local context, and reflexivity [14, 15]. PHR has been defined as a “transformative research paradigm that bridges the gap between science and practice through community engagement and social action to increase health equity” [16, p.40]. It is also a field of research that goes by different names (community-based research, action research, community-based participatory research, participatory methods, etc.), reflecting different country and ideological and disciplinary traditions .
3.2. Points of Divergence between PHR and IR
Implementation researchers are encouraged to be more explicit about the “nature” of the intervention (which should also include who defines what “it” is), what underlying sociostructural determinants are at play, and how these need to be considered in planning for both horizontal and vertical scale-up of the intervention . The starting point in IR is usually innovations or interventions with some level of demonstrated effectiveness. While participation is certainly encouraged in IR, we argue that the nuances of participatory engagement processes and related measures have not received sufficient attention in IR . This matters because participant characteristics and their structural influences can affect implementation effectiveness . The focus is generally on the participation of those responsible for delivering or implementing the innovation or intervention. This tendency reflects some of the roots of implementation research (science), which stem from quality assurance initiatives and their related focus on those formally responsible for health services delivery [1, 2]. The quality assurance approach involves the plan-do-study-act cycle but PHR and IR diverge with respect to who is primarily involved in this cycle of reflexivity .
A second point of divergence concerns scale-up of the intervention. While PHR is mainly focused on the codesign of research questions, participation in implementation, and analysis processes, implementation research (or science) is usually concerned with approaches that will enhance scale-up of effective interventions. A key difference between PHR and IR is therefore the extent of emphasis on scale-up, stemming in part from the “who” is participating in IR (i.e., usually actors delivering the intervention). There are in fact many examples of PHR where the potential scalability of the innovation or intervention and the scalable unit are simply not considered throughout the research process .
3.3. Points of Intersection between PHR and IR
Where IR and PHR fundamentally intersect is in a seemingly shared desire to generate “real-world” evidence—with many social realities unfolding. Both can be viewed as a response to the limitations of the one-way pipeline model of transferring evidence-based knowledge from researcher to end-users such as community members. Without an understanding of community needs, the uptake of such knowledge can fall short. If the ultimate goal is therefore to understand and characterize these real-world and dynamic (as opposed to linear) contextual conditions under which different social actors operate and to improve knowledge use, then PHR is necessary for all implementation research efforts. Shirk et al.  argue that outcomes are affected by the degree and quality of public participation as negotiated at the outset. The research question or implementation problem is the starting point for implementation research and related methods. For example, the degree of intervention acceptability and appropriateness during implementation are key topics for interrogation in IR, requiring a robust account of the intervention embedded in its context and the use of participatory approaches [1, 23].
PHR and IR also intersect given the increasing imperative of better accounting for context to improve understanding of how and for whom an intervention works and to explain variations in implementation . Researchers are being encouraged to better theorize about and measure context rather than treating it as something that is static and must be controlled for or as rationale for an intervention’s implementation failure [25–27]. The ways in which PHR can best inform IR therefore concern who is involved and engaged in identifying salient dimensions of context and how these may drive inequities or create barriers to or enablers of change [3, 28]. The notion of adaptation of the intervention to context is also noteworthy. This is a feature of IR, although preoccupations with questions of fidelity also present in IR may limit the extent to which consideration is given to adaptive designs [2, 28]. To date, there has been a tendency for the salient dimensions of context and the nuanced characteristics of that context to be defined by researchers, sometimes in a way that is bereft of any authentic community engagement processes . PHR approaches have also been promoted as a strategy to reduce health inequities , given some evidence that these approaches can improve social capital and cohesion . The use of PHR approaches could help in better elucidating how an intervention adapts to its context throughout implementation and with what health and health equity impacts. These approaches can inform what the intervention actually entails, how it interacts with context in compatible and incompatible ways, how and why it changes over time, and “who” is affected disproportionately by social and health inequities potentially arising from the intervention’s implementation ; they can also help better characterize and contextualize changes in coverage and benefits for intended participants . Another area for further exploration might be how PHR can contribute knowledge about why the “fidelity” of an intervention is not retained or how and why interventions are adapted when implemented in a different setting.
3.4. Who Are the “Participants” in IR Studies and Is Community Engagement Always Necessary in IR or Not?
At its core, PHR requires the democratic participation of different actors (which vary depending on the intervention and the context)—researchers, policy-makers, health system practitioners, community members, citizens, and so on. “Who” participates, the extent to which they participate, and how and when they participate are all clearly pertinent to the operationalization of PHR approaches. This leads us to also ask whether or not community engagement (and also what is meant by community) is always necessary in IR. As described above, who is engaged and the nature and the extent of community engagement are points of divergence between PHR and IR. While considered an essential element of PHR, in IR studies, community engagement is not always considered essential and how community engagement is operationalized varies considerably. With PHR, we are most often referring to the main beneficiary of the intervention (e.g., community members, target population), while, in IR, the engagement may be policy stakeholders and those responsible for delivering the intervention. Second, the timing and frequency of that engagement also vary considerably between PHR and IR.
3.5. What Are the Possible Unintended Consequences of Participatory Processes in the Context of IR Studies?
Negotiating community and research perspectives and interests present ongoing conundrums in the application of participatory approaches in research, including but not limited to IR studies. Different career and cultural imperatives between academic and community partners require open debate and transparent discussion throughout the research process [32, p. 300]. “Involving people in decisions that affect them is justified both by ethical and political arguments and by instrumental arguments asserting that involvement will lead to decisions more relevant to the people being served” [33, p. 6]. While encouraging community engagement through the use of participatory processes comes with certain advantages, it can also present risks of potential harm and unintended consequences. Authentic engagement of communities does not always transpire in all research endeavours, let alone in implementation research. Engagement in research can make communities more vulnerable. Without the necessary safeguards, such as mechanisms of co-governance that ensure transparency and accountability in decision-making, communities can inadvertently become the objects of research agendas to the benefit of others. Katz et al.  documented harms related to, for example, delegated control (e.g., participatory process perpetuates status quo of neoliberalism, colonialism, and/or racism), demobilization (e.g., participant burn-out), or sanctions (e.g., participant resistance results in loss of paid work). These authors concluded that participation was often discontinued at implementation, making it challenging for communities to hold the conveners accountable, further underscoring differences in power relations . If “communities” do know best and are well-positioned to contextualize an intervention by “integrating cultural values and practices to enhance sustainability when grant funding ends,” they also need to be valued as equal contributors in the production of knowledge . This implies that, throughout the research, all partners be engaged in how the process is governed, recognizing the unique strengths that each can contribute while mitigating any potential harms from participation. Let us take the example of North-South partnerships in global health research.
3.5.1. Power Relations between the “Participants” in the Research Process
The development and sustainability of Canada and low- and middle-income country (LMIC) partnerships that are egalitarian have been an objective of research funding in global health such as Canada’s former Global Health Research Initiative . It is, however, no easy feat and may in fact not be fully realized in practice given preexisting power asymmetries within teams, including those between researchers from different disciplines and decision-makers and communities, and across country contexts. Not only are preexisting power differences challenging to overcome in North-South partnerships, but they can also inadvertently undermine the ability to achieve an equitable orientation in research programming . In an analysis of a Canadian-led global health research funding program, the authors argued that “the donor’s finance and grant administration systems favored lower-risk Canadian institutions over generally higher-risk LMIC grantees and placed less restrictions on Canadian institutions than it did on LMIC ones” [36, p8]. Although inequitable fiscal relationships may adversely affect partnerships, other features of international partnerships can thrive. For example, an analysis across several case studies of North-South partnerships found that the global health partnered research could result in thriving and impactful partnerships. These partnerships were characterized as follows: (1) long-term and sustainable North-South partnerships; (2) interdisciplinary responses to complex issues; (3) participatory action research that grounds the research in its context; and (4) research with a policy or practice impact orientation . Bearing the above intersections between IR and PHR in mind, we now explore the role and selected practices of research funders, and in particular CIHR.
3.6. Role and Practices of Research Funders
Research funders such as CIHR can incentivize different types of research such as implementation research through priority-driven funding calls. They can also encourage the ways research should be conducted through the use of specific eligibility and peer review criteria. Funders can build incentives to help make participation more feasible and equitable through a policy of release time allowance for knowledge users such as community members and decision-makers . Tables 1 and 2 feature illustrative examples from priority-driven CIHR initiatives where deliberate efforts were taken to create funding initiatives and related criteria that support specific aspects of implementation research/science. They do not reflect the full range of IR funding calls at CIHR. The Pathways example emphasizes authentic engagements with Aboriginal communities, guided in part by OCAP® (Ownership, Control, Access, and Possession) principles . In contrast, the HIV example is focused on implementation science across settings such as communities, prisons, and so on. The global health funding opportunities reflect an example to support IR on different health issues (e.g., maternal and child health, chronic diseases), across different disciplines, countries/regions (e.g., Sub-Saharan Africa), and sectors (e.g., health systems strengthening orientation). Through the use of criteria, funders such as CIHR are stipulating who should be involved in setting the overall priorities for the research and governing its implementation and thus implicitly encouraging the use of PHR methods without actually dictating which methods should be applied. Nevertheless, for some initiatives, a wide range of appropriate research designs and disciplines that may be considered are encouraged to pave the way for applying PHR approaches. In the context of research with vulnerable populations in Canada (e.g., Aboriginal communities) and in low- and middle-income countries where there is a history of misconduct in research perpetuating colonial practices, specific criteria may be even more essential. These need to be complemented by other mechanisms (e.g., guidelines, community ethics boards) to help ensure the ethical conduct of research.
What remains an open question is the extent to which these funding incentives are having the intended impact—in terms of the degree and quality of participatory research processes and achieving meaningful results for communities and other participants in IR such as policy-makers and practitioners. The above featured examples raise important considerations for the monitoring and evaluation of participatory research processes in the context of research investments. Research funders such as CIHR use explicit eligibility and evaluation criteria to “steer” who ought to be involved in the research process, the approach to engagement, and how and the extent to which both the context and the intervention are described and are shaped by the engagement of different participants (e.g., communities, practitioners), while encouraging adequate documentation and reflexivity about ethical and equity considerations, including the ethical foundations for the interventions and the community engagement, extending what is typically considered in research ethics reviews . These requirements can help set the stage for reporting requirements for researchers after funding and for assessing the impact of funders’ practices.
There are of course limits to what funders can and cannot do to encourage researchers to use more participatory approaches in their research. Said funders could include grant writing proposal meetings where those most affected by the health inequities are brought into the discussions with researchers. They can then consider the option of engaging lay reviewers and asking them to specifically comment on the robustness of participant engagement approaches. For some grant reviews in the UK such as with the National Institute for Health Research (NIHR), funders are requiring a public engagement plan as part of the application for funding . For such NIHR reviews, someone on the peer review committee (a lay person) is asked to review all grants to assess them and comment on their public engagement approaches.
In this paper, we describe the intersecting relationship between IR and PHR, identifying points of convergence and divergence and also highlighting how health research funders can advance both fields using illustrative examples from CIHR led/co-led funding initiatives. We conclude with a few areas requiring further exploration in implementation research and participatory health research and in the assessment of research funding agency practices.
Going forward, there is first a need to evaluate the science of funding to determine whether peer review criteria and other requirements influence researchers to expand relevant expertise on their team, to engage communities differently and use a wider range of appropriate and relevant participatory approaches. Relatedly, deliberate comparisons of participatory approaches across IR initiatives of various research funders are needed.
Second, researchers need to better describe the breadth and scope of participatory health approaches in reporting to funders and in the literature to enhance the evidence base of PHR in the context of IR. Jagosh et al.  conducted a systematic realist review to capture evidence about the benefits of participation and, in particular, which mechanisms and contextual features influence outcomes. They found that participatory research supports health by improving “research quality, empowerment, capacity building, sustainability, program extension, and unanticipated new activities” [42, p. 337]. Other scholars have shown through a meta-analysis that there is robust evidence that community engagement interventions positively impact health but perhaps not surprisingly no one model was found to be more effective than another . Researchers need to be further encouraged to explicitly document the participatory health research approaches so the repertoire of methods that can be used is expanded and critiqued, any potentially harmful consequences are described (and better understood so they can be mitigated in the future), and the ways in which these processes do or do not enable implementation and scale-up of interventions are further elucidated.
Some of the ideas for this manuscript arose during the authors’ respective tenures as the former Deputy Scientific Director (EDR) and the former Scientific Director (NE) with the Canadian Institutes of Health Research-Institute of Population and Public Health. The views expressed herein are those of the authors and do not reflect those of CIHR and its institutes.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this article. They have no current or recent funding to declare that might influence this work.
- D. Peters, T. Adam, O. Alonge, I. Agyepong, and N. Tran, “Implementation research: what it is and how to do it,” British Medical Journal (BMJ), vol. 48, no. 8, pp. 675-676, 2014.
- N. Edwards and P. M. Barker, “The importance of context in implementation research,” Journal of Acquired Immune Deficiency Syndromes, vol. 67, pp. S157–S162, 2014.
- N. Edwards and E. Di Ruggiero, “Exploring which context matters in the study of health inequities and their mitigation,” Scandinavian Journal of Public Health, vol. 39, no. 6, pp. 43–49, 2011.
- G. Yamey, “What are the barriers to scaling up health interventions in low and middle income countries? A qualitative study of academic leaders in implementation science,” Globalization and Health, vol. 8, article 11, 2012.
- M. Cargo and S. L. Mercer, “The value and challenges of participatory research: strengthening its practice,” Annual Review of Public Health, vol. 29, pp. 325–350, 2008.
- T. Greenhalgh, C. Jackson, S. Shaw, and T. Janamian, “Achieving research impact through co-creation in community-based health services: literature review and case study,” Milbank Quarterly, vol. 94, no. 2, pp. 392–429, 2016.
- S. C. Bodison, I. Sankaré, H. Anaya et al., “Engaging the community in the dissemination, implementation, and improvement of health-related research,” Clinical and Translational Science, vol. 8, no. 6, pp. 814–819, 2015.
- Canadian Institutes of Health Research, “The four themes of CIHR-funded health research,” 2014, http://www.cihr-irsc.gc.ca/e/48801.html.
- Canadian Institutes of Health Research, “Global health initiatives,” 2015, http://www.cihr-irsc.gc.ca/e/46355.html.
- Canadian Institutes of Health Research, “Pathways to health equity for aboriginal peoples,” 2016, http://www.cihr-irsc.gc.ca/e/43630.html.
- Canadian Institutes of Health Research, “HIV/AIDS research initiative,” 2016, http://www.cihr-irsc.gc.ca/e/25832.html.
- Canadian Institutes of Health Research, “Pan-Canadian SPOR network in primary and integrated health care innovation,” 2016, http://www.cihr-irsc.gc.ca/e/49554.html.
- Canadian Institutes of Health Research, “CIHR’s framework for citizen engagement - section three: The CIHR citizen engagement framework,” 2013, http://www.cihr-irsc.gc.ca/e/41289.html.
- J. Springett, M. T. Wright, and B. Roche, “Developing quality criteria for participatory health research: an agenda for action,” Research Group Public Health Social Science Research Center Berlin (WZB), 2011, https://ideas.repec.org/p/zbw/wzbhea/spi2011302.html.
- J. Jull, A. Giles, and I. D. Graham, “Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge,” Implementation Science, vol. 12, article 150, 2017.
- N. Wallerstein and B. Duran, “Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity,” American Journal of Public Health, vol. 100, supplement 1, pp. S40–S46, 2010.
- M. Minkler and N. Wallerstein, Community Based Participatory Research for Health: Process to Outcomes, Jossey Bass, San Francisco, Calif, USA, 2nd edition, 2008.
- S. R. Chaudoir, A. G. Dugan, and C. H. Barr, “Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures,” Implementation Science, vol. 8, article 22, 2013.
- A. Haines, S. Kuruvilla, and M. Borchert, “Bridging the implementation gap between knowledge and action for health,” Bulletin of the World Health Organization, vol. 82, no. 10, pp. 724–731, 2004.
- World Health Organization, “Implementation research in health: a practical guide,” 2013, http://who.int/alliance-hpsr/alliancehpsr_irpguide.pdf.
- P. M. Barker, A. Reid, and M. W. Schall, “A framework for scaling up health interventions: Lessons from large-scale improvement initiatives in Africa,” Implementation Science, vol. 11, article 12, 2016.
- J. L. Shirk, H. L. Ballard, C. C. Wilderman et al., “Public participation in scientific research: A framework for deliberate design,” Ecology and Society, vol. 17, no. 2, 2012.
- R. E. Glasgow and D. Chambers, “Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science,” Clinical and Translational Science, vol. 5, no. 1, pp. 48–55, 2012.
- G. F. Moore, S. Audrey, and M. Barker, “Process evaluation of complex interventions: medical research council guidance,” British Medical Journal, vol. 350, Article ID h1258, 2015.
- P. Hawe, A. Shiell, T. Riley, and L. Gold, “Methods for exploring implementation variation and local context within a cluster randomised community intervention trial,” Journal of Epidemiology & Community Health, vol. 58, no. 9, pp. 788–793, 2004.
- R. Armstrong, E. Waters, L. Moore et al., “Improving the reporting of public health intervention research: Advancing TREND and CONSORT,” Journal of Public Health, vol. 30, no. 1, pp. 103–109, 2008.
- B. Poland, G. Krupa, and D. Mccall, “Settings for health promotion: an analytic framework to guide intervention design and implementation,” Health Promotion and Practice, vol. 10, no. 4, pp. 505–516, 2009.
- J. Shoveller, S. Viehbeck, E. Di Ruggiero, D. Greyson, K. Thomson, and R. Knight, “A critical examination of representations of context within research on population health interventions,” Critical Public Health, vol. 26, no. 5, pp. 487–500, 2016.
- “MRC guidelines for management of global health trials involving clinical or public health interventions,” 2017, https://www.mrc.ac.uk/documents/pdf/guidelines-for-management-of-global-health-trials/.
- N. B. Wallerstein and B. Duran, “Using community-based participatory research to address health disparities,” Health Promotion and Practice, vol. 7, no. 3, pp. 312–323, 2006.
- J. Popay, P. Attree, D. Hornby et al., Community Engagement in Initiatives Addressing The Wider Social Determinants of Health: A Rapid Review of Evidence on Impact, Experience and Process, University of Lancaster, Lancaster, UK, 2007.
- P. Schafer, “Community-academic partnerships and social change,” in Rethinking Social Epidemiology: Towards A Science of Change, P. O’Campo and J. Dunn, Eds., Springer, Dordrecht, Netherlands, 2011.
- A. O’Mara-Eves, G. Brunton, D. McDaid et al., “Community engagement to reduce inequalities in health: a systematic review, meta-analysis and economic analysis,” Public Health Research, vol. 1, no. 4, 2013.
- A. S. Katz, R. M. Cheff, and P. O’Campo, “Bringing stakeholders together for urban health equity: hallmarks of a compromised process,” International Journal for Equity in Health, vol. 14, article 252, 2015.
- E. Di Ruggiero, C. Zarowsky, J. Frank et al., “Coordinating Canada’s research response to global health challenges: the global health research initiative,” Canadian Journal of Public Health, vol. 97, no. 1, pp. 29–31, 2006.
- I. Daibes and S. Sridharan, “Where theory and practice of global health intersect: The developmental history of a Canadian global health initiative,” Global Health Action, vol. 7, no. 1, Article ID 23974, 2014.
- Z. Boutilier, I. Daibes, and E. Di Ruggiero, “Global health research case studies: Lessons from partnerships addressing health inequities,” BMC International Health and Human Rights, vol. 11, supplement 2, 2011.
- Canadian Institutes of Health Research, “Glossary of funding-related terms (release time allowance),” 2016, http://www.cihr-irsc.gc.ca/e/34190.html#releasetimeallowance.
- First Nations Information Governance Centre, “The First Nations Principles of OCAP®,” 2017, http://fnigc.ca/ocap.html.
- Canadian Institutes of Health Research, “CIHR team grant: pathways implementation research team – component 2,” 2016, https://www.researchnet-recherchenet.ca/rnr16/vwOpprtntyDtls.do?prog=2357&view=currentOpps&type=EXACT&resultCount=25&sort=program&all=1&masterList=true#eligibility.
- National Institute for Health Research, “Patient and public involvement at central commissioning facility,” 2017, https://www.nihr.ac.uk/about-us/how-we-are-managed/managing-centres/nihr-central-commissioning-facility/ccf-ppi/.
- J. Jagosh, A. C. MacAulay, P. Pluye et al., “Uncovering the benefits of participatory research: implications of a realist review for health research and practice,” Milbank Quarterly, vol. 90, no. 2, pp. 311–346, 2012.
- A. O'Mara-Eves, G. Brunton, S. Oliver, J. Kavanagh, F. Jamal, and J. Thomas, “The effectiveness of community engagement in public health interventions for disadvantaged groups: A meta-analysis,” BMC Public Health, vol. 15, no. 1, article no. 127, 2015.
Copyright © 2018 Erica Di Ruggiero and Nancy Edwards. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.