Objectives. To undertake a critical review describing key strategies supporting development of participatory research (PR) teams to engage partners for creation and translation of action-oriented knowledge. Methods. Sources are four leading PR practitioners identified via bibliometric analysis. Authors’ publications were identified in January 1995–October 2009 in PubMed, Embase, ISI Web of Science and CAB databases, and books. Works were limited to those with a process description describing a research project and practitioners were first, second, third, or last author. Results. Adapting and applying the “Reliability Tested Guidelines for Assessing Participatory Research Projects” to retained records identified five key strategies: developing advisory committees of researchers and intended research users; developing research agreements; using formal and informal group facilitation techniques; hiring co-researchers/partners from community; and ensuring frequent communication. Other less frequently mentioned strategies were also identified. Conclusion. This review is the first time these guidelines were used to identify key strategies supporting PR projects. They proved effective at identifying and evaluating engagement strategies as reported by completed research projects. Adapting these guidelines identified gaps where the tool was unable to assess fundamental PR elements of power dynamics, equity of resources, and member turnover. Our resulting template serves as a new tool to measure partnerships.

1. Introduction

The creation and timely translation of action-oriented knowledge can rest on meaningful engagement with end-users, even before the research begins [1, 2]. Participatory research (PR) (following Cargo and Mercer [3] and Green et al. [4] and we use PR as an umbrella term to include all partnered research, including community-based participatory research (CBPR), action research, participatory action research, participatory evaluation, community engagement and patient engagement), and community engagement continue to attract increased attention as an approach to research, requiring formation of teams of researchers in partnerships with those affected by the issue under study in the community [35] and those who will utilize the results to effect change [6, 7]. Overall, the literature suggests that the PR partnership approach increases the relevance of research questions [3, 5, 8], with the potential for effective knowledge translation [9, 10], leading to faster uptake of evidence into practice [11]. For these reasons research granting agencies, including the National Institutes of Health (NIH), the Patient Centered Outcomes Research Institute (PCORI), and the Canadian Institutes of Health Research (CIHR), are increasingly requiring that researchers partner with community members, patients, health professionals, health organisations, and policy makers, resulting in many more researchers adopting a participatory approach.

In 1995, Green and colleagues developed guidelines intended to allow reviewers of funding agencies to assess stakeholders’ engagement in PR projects [4, 12]. In 2008, these guidelines were further refined and reliability was tested to develop the Reliability Tested Guidelines for Assessing Participatory Research Projects [13] as a tool to (i) help funding agencies and peer reviewers to assess the participatory nature of proposals submitted for funding as participatory research; (ii) aid evaluators in assessing the extent to which projects meet participatory research criteria; and (iii) assist researchers and intended users of the research (i.e., nonacademic partners) in strengthening the participatory nature of their project proposals and applications for funding [12, 13]. In 2009 van Olphen et al. [14] applied these guidelines for the first time, to a single project to assess to what extent their research was participatory as perceived by community, advocacy, and scientific partners. The authors concluded that this had been a very useful undertaking and that “further research should focus on the adaptation of PR principles to assist in evaluating the process and outcomes of PR [14].”

As the principles of the PR approach are used in a wide variety of research and contexts, there is a need to explore the following questions: What are the key processes of PR and what are the practical ways to achieve equitable partnerships? What processes support the constant negotiation between all team members for research goals and objectives, partner roles and responsibilities, and decision-making procedures, together with balancing knowledge generation with the need for action? Therefore, the purpose of this study is to build on recommendations [14] and use the 2008 Reliability Tested Guidelines to undertake a critical literature review of PR projects to synthesize key practical strategies that foster a successful PR process, resulting in continuous discussions between partners that will in turn facilitate knowledge translation activities throughout the research [15].

2. Materials and Methods

2.1. Data Sources

A critical review goes beyond the description of primary studies and includes an empirical analysis for exploring new ideas [16]. While critical reviews are criticized for their nonsystematic approach, “the ‘critical’ component of this type of review is key to its value” [16].

To begin, a multidisciplinary bibliographic database (ISI Web of Science) was searched using the phrase “participatory research” for all articles from 1995 (when the initial PR guidelines were published) until October 2009 (which was the year after the Reliability Tested Guidelines were published). Results of this search yielded 1866 publications. These were then imported into CiteSpace—a bibliometric network analysis tool (http://cluster.cis.drexel.edu/~cchen/citespace/)—which generated a map of author-citation frequency. Results contained foundational PR scholars such as Paulo Freire and theoreticians including Peter Reason as well as those with practical PR experience. Our selection tool eliminated theoretical/foundational authors and retained only authors that have conducted practical PR studies. For this review we needed to limit the size of the study and chose to retain only the top four leading PR practitioners using their CiteSpace centrality scores: Barbara A. Israel, Meredith Minkler, Nina Wallerstein, and Ann C. Macaulay (Figure 1).

Next, a librarian-mediated search was conducted for all published materials by these four authors in PubMed, Embase, ISI Web of Science, PsychInfo, and CAB (Ovid database) for abstracts between January 1995 and October 2009. In addition we also reviewed chapters from books edited by these authors [1719]. Duplicates were removed, for a total of 151 records (title, authors, source, and abstract).

2.2. Study Selection

A staged selection process was then completed to limit the sample using eligibility criteria. First, records were excluded when one of the abovementioned PR leaders was neither one of the first three authors nor the last author () to ensure that the leader had substantive input into the work. The second step excluded records that were not PR related (). The third step excluded records that did not contain any description of the PR process () or records that contained only the theory of PR (). In the final step, records were excluded when they did not contain useful excerpts (), leaving 54 retained records (Figure 2).

2.3. Data Extraction

We conducted a deductive qualitative thematic analysis to extract useful data from our sample of documents [20]. For each of the 54 retained documents, relevant excerpts were selected and compiled in a Word document and organized by theme. These themes were derived from the partnership-related dimensions of the Reliability Tested Guidelines for Assessing Participatory Research Projects [13]. These guidelines contain 25 questions, 21 of which target the PR partnership process, making them very suitable to serve as themes for data extraction and analysis. These questions informed our coding scheme to identify PR process strategies. Using a coding grid based upon these questions (Table 1), partnership process-specific excerpts from the retained documents were extracted for analysis. Each retained document was reviewed in its entirety, and all excerpts in those documents that directly answered one of the questions were extracted and compiled in a matrix of “data by theme” for further analysis. Data coding was nonexclusive, and each excerpt could be coded to one or more questions on the coding grid.

2.4. Data Analysis

Data abstraction and coding were undertaken by one author (David Parry) using nonspecialized software (MS Word), which is appropriate for a deductive qualitative data analysis using a limited number of themes (codes). Each excerpt extracted from the retained documents was assigned to one or more themes, which was verified by a second author (Pierre Pluye or Jon Salsberg). Disagreements were discussed for possible resolution, and any that could not be resolved were adjudicated by a third party (Jon Salsberg for Pierre Pluye and vice versa). Using a constant comparative technique, themes were collapsed into overarching categories. These categories were generated through initial and focused coding techniques by comparing and contrasting text segments and sorting codes into conceptually meaningful units [21]. For example, subthemes such as “advisory committee,” “steering committee,” and “planning committee” were all grouped under the main theme “committee.”

3. Results

Table 2 presents the references of the 54 documents that were retained for analysis and are organized by the four main authors. From these documents, 186 excerpts were assigned or coded to one or more than one theme. Of those, there was agreement between the reviewers for 180 (97%) of excerpts. For the six remaining excerpts where there was disagreement, consensus was reached on five, and final judgment was sought from a third author (Jon Salsberg) for one excerpt.

The five most frequently mentioned strategies for fostering a researcher-community partnership are listed (unranked) and described in Table 3. These are forming an advisory board, developing a research agreement, using group facilitation techniques, hiring from the community, and having frequent meetings.

The remaining less frequently mentioned strategies are summarized in Table 4, which we felt could not be collapsed into categories without losing individual substance. However we consider these examples as also being extremely important for researchers to put into practice, including the need for researchers to make active efforts to reach out and learn about their partners and their communities; facilitating engagement by being flexible and working around schedules of the partners; understanding community priorities and culture; establishing clear lines of communication; speaking frankly and agreeing to disagree; building community capacity; supporting partners interpretation of data; publishing results in community; including nonacademic partners as copresenters and coauthors; working with community partners to build resources based on results; using the results to influence policy; and regular evaluation of the partnerships.

4. Discussion

This is a first step in a larger research agenda to identify variation in PR practices across contexts and partnership stages that could in the future be drawn on to answer the question of efficacy of PR practices. As this review was exploratory and not systematic, we decided to include a purposeful sample of included studies. CiteSpace helped us to elicit a criterion for a purposeful sampling. The rationale was that most cited papers for our review played a role similar to “key informants” in primary research. Given that this study had limited resources, we focused on the top four authors (most popular “key information resources”).

From the four authors identified, committees such as steering committees and advisory committees are the most frequently mentioned strategy as a way to engage key stakeholders around the table from the beginning—including patients, practitioners, service managers, communities and the public, and policy makers. The second most frequently mentioned strategy is drafting research agreements, which some recommend should be done early in the partnership in order to avoid misunderstandings and because the process of developing written agreements or partnership principles is in itself a partnership building process [72, 74]. However, the authors of this paper are also aware of teams who have not wanted a written agreement, either for cultural reasons where a verbal agreement is deemed very final, or due to the fact that it could be construed to imply lack of trust between the researchers and the partners. Our review results show that group facilitation is often suggested as a way to offer equal opportunity for partners to participate in discussions and to afford more reserved partners the chance to voice their opinion. Facilitation includes informal group discussions and formal techniques with many techniques borrowed from management. Hiring staff from the community increases credibility of the research, adds cultural relevance, builds capacity, promotes empowerment, provides work, brings in finances, and integrates knowledge translation throughout the process. Finally, frequent meetings are essential to maintain open communication as research evolves and to manage different expectations.

Table 4 shows many other additional practical strategies and supports the importance of meeting the needs of various partnerships in a wide range of contexts. It also emphasises the need for researchers to learn more about community issues and fully engage community members throughout the research process including interpretation of data and dissemination of results both internally within the community where the research was undertaken and externally.

To our knowledge, this is the first time the Reliability Tested Guidelines have been used to undertake a critical literature review to document PR partnership processes. The strengths of this review include (i) using a bibliometric methodology to identify leading PR practitioners, (ii) a comprehensive identification of PR studies conducted by these authors, (iii) a transparent selection of relevant documents describing PR partnership processes, and (iv) a reproducible deductive qualitative data thematic analysis using the Reliability Tested Guidelines as basis for a coding scheme to analyze relevant excerpts from retained documents. This critical review has also identified that the four authors reviewed utilise these processes and also reestablished the Reliability Tested Guidelines as reliable criteria by which to measure partnerships.

It is noteworthy that four or fewer excerpts were identified for the following Reliability Tested Guidelines dimensions: mutual learning (Q8), conflict resolution over interpretation of results (Q14), and data ownership and sharing (Q15). This is surprising considering that mutual learning is a fundamental PR principle and the latter two are key issues to be resolved for any PR project. More literature on these topics would be very useful; for example, Jagosh and colleagues found that successful conflict resolution led to further strengthening of the teams [75].

This review also highlights gaps that the Reliability Tested Guidelines do not address. These include (i) the issues of power dynamics and recommendations for ways of decentralizing power and decision-making either through subcommittees or through a high level of local control, (ii) ways to address issues of equity of resources, that is, equitable sharing of resources across community organizations and researchers, or providing grants or other funding to participating community-based organizations, and (iii) the common problem of adding or replacing new members throughout the project—which causes shifting group dynamics. We also recognize that more other human aspects of partnerships have not been addressed, including the time needed to consolidate partnerships, issues of power differences, personality clashes, and institutional cultures.

There is much diversity in the strategies discussed by the four PR leaders. This is particularly encouraging for three reasons. First, it suggests that PR is highly adaptable to many contexts and settings and the iterative nature of this research approach. As PR is rapidly expanding beyond its earlier application in health promotion with marginalized communities, this adaptability will become increasingly important for partnerships with new types of communities including communities of practice and organizations such as practice-based research networks [76] and also for partnering with patients and policy makers. Second, research teams can find many strategies in the results to draw upon when starting out. A given strategy does not always work for a given context and the whole team can discuss potential alternative strategies. Third, the diversity of results reinforces the notion that the PR process is an active, iterative endeavour, requiring energy and flexibility from all partners. The findings are supported by other authors including a critical review by Cargo and Mercer [3] and incorporated by Wallerstein and Duran [10] in a conceptual logic model of community-based participatory research. For those embarking on PR there are recommendations and training curricula from individual teams [7678] and organisations [74, 7981] on how to build PR teams and maintain equitable partnerships throughout the research process, including dissemination of the results. There are also an increasing number of publications on the experiences of both academic [82] and community [83, 84] team members from their participatory research experiences and documented common characteristics of successful community-institutional partnerships [85].

While this review provided an innovative synthesis of key PR strategies for researchers using a PR approach, a limitation is that it is based on only four authors’ publications. Because the review included book chapters not limited to the word count restrictions of journal articles we may have captured more details than from journal articles alone. There are no standard recommendations for reporting on PR; from this review we recommend that journal editors require the key stages from the Reliability Tested Guidelines to be included, which would facilitate future synthesis. Our results consist of strategies that could be tested and explored in greater detail through a larger systematic literature review, which may include more detailed descriptions of applied strategies for planning and sustaining PR partnerships. Such a systematic review might be able to rank these strategies in terms of their effectiveness in different contexts, which would first require further basic research into the efficacy of particular participatory strategies and their effectiveness in generating and translating new knowledge into action. As PR is becoming more accepted, this new evidence is slowly emerging within the fields of participatory research as well as in implementation and translational science.

5. Conclusion

This review is the first to adapt the Reliability Tested Guidelines for Assessing Participatory Research Projects to identify leading processes that support PR partnerships. Five key practical strategies to foster a successful PR process are identified that in turn integrate knowledge translation throughout the research process. Some of these results have already been incorporated into the Canadian Institutes for Health Research (CIHR) Guide to Researcher and Knowledge-User Collaboration in Health Research [81]. One colleague remarked, “I will print these 5 strategies in big color letters and pin them in front of my desk. No one can remember 25 questions, while anybody can handle 5 ideas per day.” The guidelines, originally intended to allow funders to assess partnership engagement in grant applications, proved effective at identifying and evaluating the same engagement strategies as reported by completed research projects. Adapting these guidelines for our use identified gaps where the tool was unable to assess the fundamental PR elements of power dynamics, equity of resources, and member turnover. Our resulting template serves as a new tool for research teams to apply to measure their own partnerships.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.


This work was partially supported through a grant from the Lawson Foundation, ON, Canada.