Evaluation of Psychometric Properties of a New Research Instrument for Measuring Collective Impact Based Cross-Sectoral Collaboration and Leadership: Oulu Collective Impact Study

Takalo, Tuula; Laine, Tapani; Hakko, Helinä; Niemelä, Mika; Räsänen, Sami

doi:https://doi.org/10.1155/2023/2516746

Health & Social Care in the Community

On this page

Abstract Introduction Methods Results Discussion Limitations Conclusion Data Availability Conflicts of Interest Authors’ Contributions Acknowledgments Supplementary Materials References Copyright Related Articles

Research Article | Open Access

Volume 2023 | Article ID 2516746 | https://doi.org/10.1155/2023/2516746

Evaluation of Psychometric Properties of a New Research Instrument for Measuring Collective Impact Based Cross-Sectoral Collaboration and Leadership: Oulu Collective Impact Study

Tuula Takalo,¹Tapani Laine,¹Helinä Hakko,^2,3Mika Niemelä,^2,3and Sami Räsänen^1,2

Academic Editor: Qing-Wei Chen

Received17 Apr 2023

Revised25 Oct 2023

Accepted31 Oct 2023

Published15 Nov 2023

Abstract

Cross-sectoral collaboration is a widely used strategy in social and healthcare services. The main purpose of this way of working is to bring together organizations and their expertise to solve defined problems. A collective impact (CI)-based model is one of the promising approaches to create prerequisites for functional collaboration between different sectors. The CI-based cross-sectoral collaboration model has been implemented in the City of Oulu since the year 2019. During the implementation processes as well as underlined in the literature, the role of effective leadership has been highlighted as an essential prerequisite for the successful realization of CI-related work. Furthermore, the need for a research instrument to measure the realization of cross-sectoral collaboration including different CI-based domains and the effectiveness of leadership has been emphasized. The aim of this study was to evaluate the psychometric properties of a new research instrument developed for this purpose, named the Oulu cross-sectoral Collaboration and Leadership (OCL) scale. The OCL scale is based on the CI approach, with leadership as an additional domain. The psychometric properties of the OCL scale were evaluated for validity and reliability. Two-factor analyses with varimax rotation were performed to verify that the items included in each domain formed a homogeneous entity. The number of respondents in the online survey was 254, covering 37% of the target group of professionals. The Cronbach’s alpha of the seven domains ranged from 0.875 to 0.929, reflecting good to excellent internal consistency. The results of the factor analyses showed that the items in all the domains of this research instrument work together as a group in an appropriate manner, measuring the same object. Suggestions for further development of the instrument are also presented. The OCL scale proved to be a promising tool for research purposes and for assessing collaborative development work of public services. This also gives an opportunity to measure changes in cross-sectoral collaboration during the CI-based implementation process.

1. Introduction

The Finnish social and healthcare service system is based on public healthcare services to which everyone residing in the country is entitled. It is internationally considered to be both effective and efficient [1]. However, over the past few years, it has not been able to respond satisfactorily to an increasing demand for services. One fundamental reason for this imbalance is stated to be the fragmented and isolated entities of service organizations [2–4]. Furthermore, decentralized leadership and decision-making have been thought to have a deteriorating impact on the efficiency of social and healthcare services [2].

Cross-sectoral collaboration is a widely used strategy with plenty of theories and models. These aim to solve the fragmentation of services and to improve dealing with complex and intertwined social and health problems. The main purpose of this way of working is to bring together organizations and their expertise to solve defined significant societal and health problems [5, 6]. Despite various programs launched to enhance cross-sectoral collaboration, the service practices have remained somewhat isolated, and therefore, for example, the use of institutionalized social and healthcare services has remained at a high level [7–11].

The collective impact (CI) approach is demonstrated to be a promising tool to improve cross-sectoral collaboration [12–14]. The CI approach is used to enhance cross-sectoral collaboration between public services to respond to the complex health and social problems of children, adolescents, and families [12, 15–17], such as nutrition, social and health problems, homelessness, and education-related challenges [15, 18, 19]. The structure of this approach is based on five separate entities, termed key conditions by Kania & Kramer [12]: a common agenda, shared measurement system, mutually reinforcing activities, continuous communication, and backbone support organization. In addition, the role of leadership has been emphasized as an additional prerequisite for the successful implementation of CI-based work in everyday working practice [20–22].

In Finland, CI framework-based models have been implemented in some municipalities from the year 2013 onwards [13, 14]. The models have aimed to enhance cross-sectoral collaboration between daycare, education, and social and healthcare services. The ultimate aim of these models has been to prevent the escalation of both health and social problems and, hence, to reduce the need for curative services for children. In the City of Oulu in Northern Finland, a cross-sectoral collaboration model based on the CI approach has been implemented in the welfare service areas since the year 2019 [14]. During these implementation processes, the need for a research instrument to measure the realization of CI-based cross-sectoral collaboration and its leadership was emphasized. However, after reviewing previous literature to find an appropriate CI-based assessment tool, some CI-related surveys utilizing questionnaire-based data were found, e.g., [23–27], but those instruments did not appear to be comprehensive enough for the current CI research context. For example, a survey instrument of Salignac and workgroup [24] addressed only health-related collaboration but did not include other service sectors or CI-related leadership. In summary, the prior instruments lacked child and family service context, leadership, or CI conditions-based items.

In order to evaluate the success of the implementation of the CI-based cross-sectoral collaboration model comprehensively in working practice, a new research instrument, named the Oulu cross-sectoral Collaboration and Leadership (OCL) scale, was developed by the research group. This instrument consists of seven domains; of these, six were based directly on CI conditions and one additional new domain addresses leadership as noted in discussions of earlier studies [20–22]. The preliminary OCL scale was thoroughly reviewed by a selected external group of professionals. After that, an online survey using this OCL scale was performed among the target group comprising professionals working in public health sectors for social care, healthcare, and education services in the city of Oulu. The current study evaluates the psychometric properties of this new OCL scale. Suggestions for further development of the instrument are also presented.

2. Methods

2.1. Theoretical Background of the New Research Instrument

The OCL scale includes seven domains, six of which are based directly on the five conditions of the CI approach [12]. In addition, leadership was included in this scale as a seventh domain because the role of leadership is increasingly highlighted in the literature as an essential prerequisite for the successful realization of CI-related work [20–22, 28]. The description of the domains and their theoretical context are presented in Table 1.

2.2. Development Process of the OCL Scale

The development process of the OCL scale is visualized in Figure 1. In the first phase, five conditions of the CI approach and the leadership were operationalized into the items by two members of the research group (TT and MN) with long-term experience and extensive knowledge of work using the CI approach. The CI condition for the common agenda was assessed in the OCL scale using two separate domains. It was considered important to get distinct information on the strategy level (i.e., how the shared goal of the cross-sectoral collaboration was accepted) and operationalizing level (i.e., how the cross-sectoral collaboration-related goal was carried out in everyday work). Thus, a total of seven separate domains (entities of items) were included in the OCL scale. These domains comprised a total of 47 items scored with a forced Likert-type scale from 1 (strongly disagree) to 4 (strongly agree). The number of items in each domain is as follows: shared operating model, strategy level (8 items); shared operating model, action level (9 items); follow-up (6 items); practices that support (6 items); shared communication (5 items); backbone support structure (6 items); and leading (7 items). The OCL scale is attached as supplementary material.

In the pilot phase, the preliminary OCL scale was tested and reviewed by a selected external group of five professionals from social care, healthcare, and education services who had long-term experience in cross-sectoral collaboration work. Written feedback on the preliminary OCL scale was collected and checked thoroughly by the research group. The external group of professionals considered the OCL scale appropriate for the intended purposes.

2.3. Carrying Out the Survey

The online survey utilizing the final version of the OCL scale was conducted using the Webropol survey and reporting tool [29]. The survey was open during two time periods: April 7th, 2021–May 10th, 2021, and September 24th, 2021–November 17th, 2021.

The target group of the online survey using the OCL scale comprised all professionals (n = 683) who were working in social, healthcare, and education services in the city of Oulu and were involved in concrete cross-sectoral collaboration work in their service sectors (Figure 2). Some of the potential participants working in the education sector were also in a leadership position (n = 97), and they had personal experience of concrete cross-sectoral collaboration, like arranging and participating in network meetings. Altogether 269 professionals responded to the online survey. Of them, 15 were excluded from further analyses because their working unit had changed, and their current occupation did not meet the inclusion criteria of the initial target group of the study. Thus, the final number of respondents was 254, covering 37% of the target group.

2.4. Statistical Methods

Data were mainly analyzed using the IBM SPSS Statistics, version 28; confirmatory factor analysis was conducted by utilizing R for statistical computing, version 4.2.3, and its lavaan package.

In this study, the psychometric properties of the OCL scale comprised evaluation of the internal consistency of the items, assessment of the validity of the theoretical model in the dataset, and re-exploration of the factor structure of the OCL scale. The detailed results of these analyses are summarized in Supplementary Tables 1 and 2.

First, the items of the seven domains of the OCL scale were evaluated for validity and reliability using the Cronbach’s alpha statistics [30]. Values above 0.9 are considered to describe excellent, 0.8 to 0.9 good, 0.7 to 0.8 acceptable, 0.6 to 0.7 questionable, 0.5 to 0.6 poor, and those below 0.5 unacceptable internal consistency of the items. The evaluation of Cronbach’s alpha if an item of a domain was deleted was used to assess whether an item should be removed to increase the internal consistency of a domain.

Sample adequacy was assessed using the Kaiser-Meyer-Olkin (KMO) statistic and Bartlett’s sphericity test. The KMO statistic was used to assess whether the sample size was sufficient for factor analysis. A KMO statistics value closer to 1.0 indicates better factor analysis results, while values below 0.6 suggest that factor analysis is not appropriate. Bartlett’s sphericity test was used to assess whether the correlations of items in the data are strong enough to use a dimension-reduction technique. The null hypothesis of Bartlett’s sphericity test is that the items are not correlated. Furthermore, the adequacy of sample size was determined in terms of the study’s subject to item ratio 5 : 1 and using the recommendation of N ≥ 200 for the theoretical model [31–33]. In the current study, the sample size (N = 254) and the ratio of sample size to number of items (n = 47) of the scale were acceptable (5.4 : 1).

Confirmatory factor analysis (CFA) was used to test the validity of a theoretical model by analyzing the hypothesized relationships among the items of the seven domains of the OCL scale. The following model fit statistics (and in parenthesis, criteria for acceptable level) were used to quantify the goodness of fit or the deviance from the perfect model fit in the dataset of our study: minimum discrepancy function by degrees of freedom divided (χ²/df) (≤3 = acceptable fit, ≤5 = reasonable fit), Comparative Fit Index (CFI) (>0.9 acceptable fit, value between 0.8 and 0.9 = moderate fit), Tucker–Lewis Index (TLI) (>0.9 = acceptable fit, value between 0.8 and 0.9 = moderate fit), root mean square error of approximation (RMSEA) (≤0.08 = acceptable fit, value between 0.08 and 0.10 = moderate fit), and standardized root mean squared residual (SRMR) (≤0.08 = acceptable fit) [34–36]. However, it is recommended that the cut-off criteria of these fit indices should be interpreted as rules of thumb rather than golden rules for model fit [37].

Explorative factor analysis (EFA) with the varimax rotation method was used to assess the factor structure of the items in our dataset without a priori assumption of a certain structure or number of latent factors [38]. A standardized factor loading of greater than 0.40 was considered to indicate a moderate correlation between the item and the underlying latent factor. The factor structure based on the EFA was further evaluated with the same CFA using model fit and Cronbach’s alpha statistics as was done in CFA for the theoretical model.

To visualize the relationship of the theoretical model built based on CFA to the EFA-based model, Sankey diagrams [39] were utilized for visualization. On the left side of the diagram, the seven domains with the number of items in each domain represent the theoretical factor structure of the OCL scale. The width of each flow indicates the stability/nonstability of factor structure, i.e., the proportion of items that “change place” or remain stable when comparing the theoretical CFA model to the factor structure of the EFA-based model (right side of the diagram) [40]. Figure 3 is created using the open-source, online, and open-access software of SankeyMATIC [39].

Figure 3

Sankey-type flow diagram visualizing the theoretical factor structure of the seven domains of the Oulu cross-sectoral Collaboration and Leadership (OCL) scale in relation to factor structure produced by the exploratory factor analysis (EFA) [39]. On the left side, the theoretical factor structure of the Oulu cross-sectoral collaboration and leadership (OCL) scale is used as starting nodes. On the right side, the ending nodes represent factors suggested by the exploratory factor analysis. The width of each flow is proportional to the number of items at starting and ending nodes.

2.5. Qualitative Analysis

The OCL scale included one open-answered question: “If you wish, you can comment on the operational models, practices, follow-up monitoring, and leading of cross-sectoral collaboration,” providing respondent a possibility for additional comments. Content analysis was used to analyze these responses. Content analysis is defined as a systematic method describing and quantifying research phenomena. It is a report method for making valid conclusions from qualitative data with the purpose of providing knowledge, new insights, a representation of facts, and a practical guide to action [41]. Content analysis includes both qualitative and quantitative approaches [42].

Based on the principles of the content analysis [41–43], the responses were thoroughly reviewed and categorized into three main categories: cross-sectoral collaboration, CI-based collaboration, and feedback on the use of research instrument (TT). In the current study, only the responses belonging to the use of the research instrument category were included for further analysis. To ensure the consistency of the content analysis [44], the other author (MN) reviewed the qualitative data and their categories. Finally, the data and the analysis process were presented to the research group to verify the consistency of the content analysis process.

3. Results

3.1. Characteristics of the Study Participants

Out of 254 survey participants, 45 (17.7%) were employed in leadership positions while the remaining 209 (82.3%) represented other professionals. More than two-thirds, i.e., 163 (64.2%) of all respondents had worked in their field for more than five years. Social and healthcare employees accounted for 34.3%, and daycare and school employees accounted for 65.7% of total respondents (Table 2).

3.2. Internal Consistency

Table 3 shows the Cronbach’s alphas calculated for each of the seven domains of the OCL scale. The Cronbach’s alpha of the total OCL was 0.968 and ranged in domains from 0.875 to 0.929, suggesting good to excellent internal consistency.

3.3. Sampling Adequacy

Both the KMO statistic (KMO = 0.943) and Bartlett’s sphericity test (χ² = 9741.8, ) indicated that the data under analysis were adequate for a factor analytic approach.

3.4. Theoretical Model, Confirmatory Factor Analysis (CFA)

CFA was utilized to assess the validity of a factor structure of the theoretical model in the data under analysis. The detailed results of CFA are reported in Supplementary Table 1. Generally, the model fit statistics for the seven-domain model suggested a moderate to acceptable fit of the data with the theoretical model: χ²/df = 2.856 (acceptable fit), CFI = 0.800 (moderate fit), TLI = 0.786 (poor fit), RMSEA = 0.085 (moderate fit), and SRMR = 0.078 (acceptable fit).

3.5. Explanatory Factor Analysis (EFA)

EFA was performed to explore the factor structure of the items in our dataset without a priori assumption of a certain structure or number of latent factors. The results of EFA and subsequent CFA are reported in Supplementary Table 2. In the EFA model, when a limit of >0.40 for eligible factor loading was applied, there were eight items that loaded rather highly to two factors. The EFA-based model was further evaluated with CFA, and the model fit statistics of this adjusted CFA model were as follows: χ²/df = 2.463 (acceptable fit), CFI = 0.842 (moderate fit), TLI = 0.831 (moderate fit), RMSEA = 0.076 (acceptable fit), and SRMR = 0.073 (acceptable fit). The Cronbach’s alphas of factors of the adjusted CFA model varied from 0.826 to 0.936 indicating good to excellent internal consistency.

3.6. Relationship between Theoretical CFA Model and EFA-Based Model

Figure 3 visualizes the association between the theoretical factor structure (CFA) and the factor structure based on EFA. The seven items of initial domain 7 for leadership remained unchanged in both CFA and EFA. In domain 5 for shared communication, the number of items after EFA modelling doubled, from 5 to 10 items, while the number of items in domain 5 for backbone support structures decreased from six to three. In other domains, there were moderate changes in items between the initial theoretical-based CFA model and the EFA-based model.

3.7. Qualitative Analysis

Altogether 19 of a total of 105 responses were related to the actual use of the research instrument. Of them, 11 suggested that the research instrument should include neutral “don’t know” or “cannot say” options in the scale:

“It would have been good to have the answer option “I don't know” in the survey. Many things were such that there is no information as to whether it works like this.” (ID 244)

In addition, six respondents commented that the survey was difficult to answer. They responded that some of the items concerned issues with which they were not familiar. They recommended a neutral or “don’t know” option because

“I had no knowledge of all the points (e.g., how much the operational model is discussed at the managerial level or whether the operational model has received external funding.” (ID 123)

Therefore, some respondents perceived that response options without the possibility to choose a “don’t know” option did not reflect their reality in working practice.

A more precise definition of cross-sectoral collaboration and the focus group was desired by two respondents:

“At the beginning, it would be good to explain what the operating model means here and how it needs to work. Now, there is an item that is too open to evaluate.” (ID 260)

“When answering the survey, my answers may be distorted by which leader is meant in the questions: my own leader is not involved in cross-sectoral collaboration.” (ID 204)

Based on the respondents’ comments, the instructions for the research instrument need to be clarified in the future. The key concepts of the instrument also require more precise definitions. Similarly, it is important to define in a more detailed manner the target group of the subjects that are intended to be evaluated with the OCL scale.

4. Discussion

Dysfunctional cross-sectoral collaboration, including fragmentation and overlap of services, is widely regarded as fundamental causes of ineffective utilization of resources, especially in dealing with complex social and health problems [2, 3, 45–47]. There have been attempts to solve this multifaceted problem with various approaches, of which the CI approach has been considered to be one of the most promising. Attention has also been focused on effective leadership, which has a key role in successful cross-sectoral collaboration. Therefore, leadership has been proposed as an eminent part of cross-sectoral collaboration-based work [48–52]. Following the principles of the CI framework together with joint leadership, including leaders from all relevant service sectors, is a promising way to increase efficiency in services [13, 14, 22, 28]. However, in terms of measuring comprehensively the success of CI-based cross-sectoral collaboration, an appropriate research instrument for evaluation purposes has been lacking. Evaluation of the implementation process and changes in the services after implementation of the CI-related approach in practices are also called for in previous literature [21, 22].

In the City of Oulu, the CI approach was implemented to enhance collaboration over sectoral boundaries. Research on the realization of cross-sectoral collaboration and leadership was combined with this implementation process [9]. The necessity for the current study arose from a practical need to evaluate the fulfillment of collaboration across the social and healthcare, daycare, and education service sectors. For this purpose, a new research instrument named the Oulu cross-sectoral Collaboration and Leadership (OCL) scale was developed based on the five CI conditions of the CI approach and with leadership as an additional domain. This study evaluated the psychometric properties of this instrument from the perspective of with and without a priori theory of its factor structure. Suggestions for and discussion about the further development of the survey instrument are presented.

The results of the psychometric analyses of the OCL scale indicated that the theoretical factor structure of the OCL scale with seven domains had moderate to acceptable fit with the data under analysis, whether internal consistency of the domains was assessed with confirmatory factor analysis (CFA) or Cronbach’s alpha statistics. Further evaluation of the factor structure of the OCL scale was performed by utilizing exploratory factor analysis (EFA) without a priori theory of the latent factors of the scale and by performing subsequent CFA based on the EFA results. The purpose of the latter analyses was to give further information on the items and their placement in the domains. The main results of the analyses indicated that the items of the new domain for leadership were stable, while notable changes in items were observed in the domains for shared communication and backbone support structure. In other domains, some moderate changes in items were observed between the theoretical and the EFA-based models.

The leadership domain as part of the CI-based cross-sectoral collaboration was included in the OCL scale because the role and importance of effective leadership in collaboration have been increasingly emphasized in the previous literature [22, 28] as well as in a published study of implementation work in the city of Oulu [14]. In this new domain of leadership, internal consistency was shown to be high, and the stability of the items was evident. This finding may indicate that from the point of view of the respondents, the leadership-related items were recognizable in everyday working practice. The qualitative responses of the survey participants also did not reveal any criticism towards leadership items as part of the research instrument. Furthermore, in our study, the items of the leadership domain of the OCL scale were shown to form their own and specific entity, because none of the items of the other domains had the potential for it. In conclusion, the results suggested that the leadership domain is a useful addition to the OCL scale evaluating the implementation of CI-based cross-sectoral collaboration work in working practice.

Despite the high internal consistency and promising results of the model fit statistics, further development of the OCL scale and its psychometric construct is worth considering in future research. For example, the results of EFA showed that eight items had rather high loading to two factors. This warrants further evaluation of the nature of the items that lead to this cross-loading. In addition, shortening the OCL scale by reducing the number of items included in the OCL scale, and its domains would be justified to make the OCL scale more feasible to complete. Although the statistics for sampling adequacy and the ratio of sample size under analysis to the number of items showed satisfactory results, further testing of the OCL scale with large samples is called for.

Some suggestions for further development of the OCL scale also arose from the qualitative responses of the instrument. The answers to the open question of the OCL scale highlighted the need to clarify the key concepts used in the OCL scale. Clarification was called for concepts such as cross-sectoral collaboration, e.g., what kind of cross-sectoral collaboration was meant in the OCL scale. In addition, the respondents suggested that the five-point Likert scale should also include a neutral (i.e., “don’t know”) option instead of the current forced four-point Likert scale used in the OCL scale. When developing the OCL scale, we considered that it was essential to get the respondents to express clear opinions on each item and, thus, a four-point Likert scale without a neutral option was chosen [53–56] However, the respondents’ criticism was partly understandable; for example, some respondents claimed to have too little information of CI-based work to make a justified choice between the four options of an item.

Previous research literature has documented that CI-based working models have been implemented and evaluated since 2011. These studies have been carried out mainly in the USA and Canada [15, 18, 19] as well as in Western Europe, e.g., in Ireland [57, 58] and Scotland (e.g., 52). However, there is a lack of systematic, CI-based survey instruments measuring achievable proceedings in child and family-centered cross-sectoral collaboration. The OCL scale, described and tested in the current study, appears to be promising as an evaluation tool in the child, adolescents, and family-related CI initiatives. Using an instrument such as the OCL scale in our study would also enable international assessments and comparisons, given that the characteristics of cross-sectoral collaboration are quite similar as described in this study, for example, how to carry out cross-sectoral leadership. In addition, it should be noted that the OCL scale was developed for evaluation of CI-based work in the Nordic welfare service systems and, thus, modifications might be needed according to other service environments. In the Nordic countries, social and healthcare service systems have many similarities with each other and a need to enhance the cross-sectoral collaboration has been called for [4, 11, 59–61]. The CI model applied in the Finnish service system [13, 14] is a feasible way to proceed in improving cross-sectoral collaboration work. In the evaluation of the effectiveness of CI-based work, the OCL scale could be an appropriate tool to measure it.

5. Strengths and Limitations

The strength of this study is that the respondents of the survey who completed the OCL scale worked by utilizing cross-sectoral collaboration in their everyday working practice, and thus, they were considered to form a homogeneous target group in terms of cross-sectoral collaboration work. The overall response rate was good (37%) considering general response rates in corresponding surveys [55, 56]. The limitation of this study was that only those leaders who had concrete psychosocial support responsibilities were included in the target group while leaders with no current practical experience concerning cross-sectoral collaboration were left out of the study. This may have caused bias while evaluating a leader’s role in cross-sectoral collaboration.

6. Conclusion

The OCL scale appears to be a promising new research instrument to measure comprehensively CI-related cross-sectoral collaboration between social and healthcare, daycare, and school services. The research instrument is based on all CI conditions and has leadership as an additional domain. The OCL scale proved to be an appropriate tool for research purposes and for assessing the development work of the services, but some modifications would improve its psychometric construct. The results of this study indicated that leadership is a key part of collaboration, easy to recognize by the respondents, and, thus, is an important addition to the research instrument. A need to clarify concepts and shorten the survey is an essential goal for further revision of the scale.

Data Availability

Data are available upon request due to privacy and ethical restrictions.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

All authors planned and performed the study design. TT and MN developed an initial survey, and TT performed the data collection with the OCL scale. HH and TL were responsible for quantitative data analysis and reporting the results. HH, TT, and MN analyzed and reported qualitative data. All authors performed an interpretation of the results based on the data analyses. TT and MN wrote the first draft of the manuscript. SR revised the manuscript and guided the process throughout the study. All authors participated in the critical drafting of the manuscript and approved the final version.

Acknowledgments

The scholarship of TT (2020-2021) and the professorship of MN (2020–2023) have been funded by ITLA Children’s Foundation. This study was supported by the Strategic Research Council (SRC) established in the Academy of Finland, decision number 352509 (Researchers HH, MN). Open Access funding was enabled and organized by FinELib 2023.

Supplementary Materials

(1a) Survey covering letter to the participants: mutual cross-sectoral collaboration between early childhood education, schools, and social and healthcare services. (1b) The original research instrument of the survey: the Oulu Cross-Sectoral Collaboration and Leadership research instrument ©. (2) Theoretical factor structure of the Oulu cross-sectoral Collaboration and Leadership (OCL) scale. (3) Factor structure of the Oulu cross-sectoral Collaboration and Leadership (OCL) scale based on the results of the exploratory factor analysis (EFA). (Supplementary Materials)

References

N. Fullman, J. Yearwood, S. M. Abay et al., “Measuring performance on the healthcare access and quality Index for 195 countries and territories and selected subnational locations: a systematic analysis from the global burden of disease study 2016,” The Lancet, vol. 391, no. 10136, pp. 2236–2271, 2018.
View at: Publisher Site | Google Scholar
I. Keskimäki, T. Sinervo, and J. Koivisto, “Integrating health and social services in Finland: regional and local initiatives to coordinate care,” Public Health Panorama, vol. 4, no. 04, pp. 679–687, 2018.
View at: Google Scholar
T. Hetemaa, R. Kannisto, N. Knape et al., “From Information to Assessment Aiming for Better Services,” Social welfare and health care services in Finland 2019, Finnish Institute for Health and Welfare, Helsinki, Finland, 2021.
View at: Google Scholar
H. Tiirinki, J. Sulander, T. Sinervo, S. Halme, and I. Keskimäki, “Integrating health and social services in Finland: regional approaches and governance models,” International Journal of Integrated Care, vol. 22, no. 3, pp. 18–11, 2022.
View at: Publisher Site | Google Scholar
L. Calancie, L. Frerichs, M. M. Davis et al., “Consolidated Framework for Collaboration Research derived from a systematic review of theories, models, frameworks and principles for cross-sector collaboration,” PLoS One, vol. 16, no. 1, Article ID e0244501, 2021.
View at: Publisher Site | Google Scholar
J. M. Bryson, B. C. Crosby, and M. M. Stone, “Designing and implementing cross‐sector collaborations: needed and challenging,” Public Administration Review, vol. 75, no. 5, pp. 647–663, 2015.
View at: Publisher Site | Google Scholar
A. Sourander, L. Lempinen, and A. Brunstein Klomek, “Changes in mental health, bullying behavior, and service use among eight-year-old children over 24 years,” Journal of the American Academy of Child & Adolescent Psychiatry, vol. 55, no. 8, pp. 717–725.e2, 2016.
View at: Publisher Site | Google Scholar
H. O. Atladottir, D. Gyllenberg, A. Langridge et al., “The increasing prevalence of reported diagnoses of childhood psychiatric disorders: a descriptive multinational comparison,” European Child & Adolescent Psychiatry, vol. 24, no. 2, pp. 173–183, 2015.
View at: Publisher Site | Google Scholar
A. Kääriälä and H. Hiilamo, “Children in out-of-home care as young adults: a systematic review of outcomes in the Nordic countries,” Children and Youth Services Review, vol. 79, pp. 107–114, 2017.
View at: Publisher Site | Google Scholar
S. K. Lipson, E. G. Lattie, and D. Eisenberg, “Increased rates of mental health service utilization by U.S. College students: 10-year population-level trends (2007–2017),” Psychiatric Services, vol. 70, no. 1, pp. 60–63, 2019.
View at: Publisher Site | Google Scholar
K. Croell, T. Tiina Hetemaa, N. Knape et al., “From information to assessment aiming for better services,” Organising of Healthcare and Social Welfare in Finland National Expert Assessment, Finnish Institute for Heaslth and Welfare specialist group, Welfare, Finland, 2023.
View at: Google Scholar
J. Kania and M. Kramer, “Collective impact,” Stanford Social Innovation Review, vol. 15, pp. 36–41, 2011.
View at: Publisher Site | Google Scholar
M. Niemelä, H. Kallunki, J. Jokinen et al., “Collective impact on prevention: let’s Talk about Children Service Model and decrease in referrals to child protection services,” Frontiers in Psychiatry, vol. 10, p. 64, 2019.
View at: Publisher Site | Google Scholar
T. Takalo, S. Räsänen, H. Hakko, A. Juutinen, and M. Niemelä, “Rationale and description of implementation of regional collaborative service model for enhancing psychosocial wellbeing,” Frontiers in Psychiatry, vol. 2022, p. 13, 2022.
View at: Publisher Site | Google Scholar
S. Panjwani, T. Graves-Boswell, W. R. Garney, D. Muraleetharan, M. Spadine, and S. Flores, “Evaluating collective impact initiatives: a systematic scoping review,” American Journal of Evaluation, vol. 44, no. 3, pp. 406–423, 2023.
View at: Publisher Site | Google Scholar
J. Flood, M. Minkler, S. Hennessey Lavery, J. Estrada, and J. Falbe, “The collective impact model and its potential for health promotion: overview and case study of a healthy retail initiative in San Francisco,” Health Education & Behavior, vol. 42, no. 5, pp. 654–668, 2015.
View at: Publisher Site | Google Scholar
L. Demant and J. Lawrence, “Back on track: the challenges of implementing a small place-based collective impact initiative,” Health Promotion Journal of Australia, vol. 29, no. 3, pp. 360–362, 2018.
View at: Publisher Site | Google Scholar
G. Ennis and M. Tofa, “Collective impact: a review of the peer-reviewed research,” Australian Social Work, vol. 73, no. 1, pp. 32–47, 2020.
View at: Publisher Site | Google Scholar
P. Virtanen, T. Ristikari, and M. Niemelä, “Partnership and backbone organizations as enablers of children’s wellbeing,” Encyclopedia of the UN Sustainable Development Goals, OSUA, Sambalpur, India, 2020.
View at: Publisher Site | Google Scholar
P. Schmitz, Everyone Leads. Building Leadership from the Community up, Jossey-Bass. A Wiley Imprint, San Francisco, CA, USA, 2012.
H. Preskill, M. Parkhurst, and J. Splansky Juster, “Guide to Evaluating Collective Impact. Parts 1 – 3. Collective Impact Forum,” 2014, http://thrivingcolorado.com/wp-content/uploads/2017/12/Evaluating-Collective-Impact.pdf.
View at: Google Scholar
S. Stachowiak, J. Lynn, T. Akey et al., When Collective Impact Has Impact. A Cross-Site Study of 25 Collective Impact Initiatives, Spark Policy Institute of Denver, CO & ORS Impact of Seatle, Washington, WA, USA, 2018.
A. Meinen, A. Hilgendorf, A. L. Korth et al., “The Wisconsin early childhood obesity prevention initiative: an example of statewide collective impact,” Wisconsin Medical Journal: Official Publication of the State Medical Society of Wisconsin, vol. 115, no. 5, pp. 269–274, 2016.
View at: Google Scholar
F. Salignac, A. Marjolin, J. Noone, and G. Carey, “Measuring dynamic collaborations: collaborative health assessment tool,” Australian Journal of Public Administration, vol. 78, no. 2, pp. 227–249, 2019.
View at: Publisher Site | Google Scholar
M. B. Hargreaves, N. Verbitsky-Savitz, B. Coffee-Borden et al., “Advancing the measurement of collective community capacity to address adverse childhood experiences and resilience,” Children and Youth Services Review, vol. 76, pp. 142–153, 2017.
View at: Publisher Site | Google Scholar
N. L. Weaver, J. Thompson, C. R. Shoff, K. Copanas, and S. E. McMillin, “A conceptual model for the pathways of effect for intermediary organizations: a case study from maternal and child health,” Evaluation and Program Planning, vol. 63, no. 1, pp. 69–73, 2017.
View at: Publisher Site | Google Scholar
S. Landry, V. Collie-Akers, K. Foster, D. Pecha, and C. Abresch, “Assessing the development of collective impact initiatives addressing maternal and child health,” Maternal and Child Health Journal, vol. 24, no. 4, pp. 405–411, 2020.
View at: Publisher Site | Google Scholar
F. Hanleybrown, J. Kania, and M. Kramer, “Channeling change: making collective impact work. Stanford social innovation review,” 2012, https://ssir.org/articles/entry/channeling_change_making_collective_impact_work.
View at: Google Scholar
O. Webropol, “Webropol,” 2023, https://webropol.fi/.
View at: Google Scholar
J. A. Gliem and R. R. Gliem, “Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales,” Midwest Research-To-Practice Conference in Adult, Continuing, and Community Education, Indianapolis, Indiana, 2003.
View at: Google Scholar
G. O. Boateng, T. B. Neilands, E. A. Frongillo, H. R. Melgar-Quiñonez, and S. L. Young, “Best practices for developing and validating scales for health, social, and behavioral research: a primer,” Frontiers in Public Health, vol. 6, no. 149, p. 149, 2018.
View at: Publisher Site | Google Scholar
E. Anthoine, L. Moret, A. Regnault, V. Sebille, and J.-B. Hardouin, “Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures,” Health and Quality of Life Outcomes, vol. 12, no. 1, p. 2, 2014.
View at: Publisher Site | Google Scholar
N. D. Myers, S. Ahn, and Y. Jin, “Sample size and power estimates for a confirmatory factor analytic model in exercise and sport: a Monte Carlo approach,” Research Quarterly for Exercise & Sport, vol. 82, no. 3, pp. 412–423, 2011.
View at: Publisher Site | Google Scholar
M. Alavi, D. Visentin, D. Thapa, G. Hunt, R. Watson, and M. Cleary, “Chi-square for model fit in confirmatory factor analysis,” Journal of Advanced Nursing, vol. 76, no. 9, pp. 2209–2211, 2020.
View at: Publisher Site | Google Scholar
D. Goretzko, K. Siemund, and P. Sterner, “Evaluating model fit of measurement models in confirmatory factor analysis,” Educational and Psychological Measurement, vol. 1, no. 0, Article ID 001316442311638, 2023.
View at: Publisher Site | Google Scholar
J. B. Schreiber, A. Nora, F. K. Stage, E. A. Barlow, and J. King, “Reporting structural equation modeling and confirmatory factor analysis results: a review,” The Journal of Educational Research, vol. 99, no. 6, pp. 323–338, 2006.
View at: Publisher Site | Google Scholar
H. W. Marsh, K. Hau, and Z. Wen, “In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing hu and bentler’s (1999) findings,” Structural Equation Modeling: A Multidisciplinary Journal, vol. 11, no. 3, pp. 320–341, 2004.
View at: Publisher Site | Google Scholar
M. Tavakol and A. Wetzel, “Factor Analysis: a means for theory and instrument development in support of construct validity,” International Journal of Medical Education, vol. 11, no. 11, pp. 245–247, 2020.
View at: Publisher Site | Google Scholar
SankeyMATIC, “Steve bogart,” 2023, https://sankeymatic.com/.
View at: Google Scholar
P. Riehmann, M. Hanfler, and B. Froehlich, “Interactive Sankey diagrams,” IEEE Symposium on Information Visualization, INFOVIS, Minneapolis, MN, USA, 2005.
View at: Google Scholar
K. Krippendorff, Content Analysis. An Introduction to its Methodology, SAGE Publishing, Los Angeles, LA, USA, 2018.
S. Elo and H. Kyngäs, “The qualitative content analysis process,” Journal of Advanced Nursing, vol. 62, no. 1, pp. 107–115, 2008.
View at: Publisher Site | Google Scholar
M. Bengtsson, “How to plan and perform a qualitative study using content analysis,” NursingPlus Open, vol. 2, pp. 8–14, 2016.
View at: Publisher Site | Google Scholar
S. Elo, M. Kääriäinen, O. Kanste, T. Pölkki, K. Utriainen, and H. Kyngäs, Qualitative Content Analysis: A Focus on Trustworthiness, SAGE, Thousand Oaks, CA, USA, 2014.
K. C. Stange, “The problem of fragmentation and the need for integrative solutions,” The Annals of Family Medicine, vol. 7, no. 2, pp. 100–103, 2009.
View at: Publisher Site | Google Scholar
Cqc, “Care, Quality and Comission. Review of children and young people’s mental health services: phase one report,” 2017, https://www.cqc.org.uk/sites/default/files/20171103_cypmhphase1_report.pdf.
View at: Google Scholar
R. McGihon, L. D. Hawke, G. Chaim, and J. Henderson, “Cross-sectoral integration in youth-focused health and social services in Canada: a social network analysis,” BMC Health Services Research, vol. 18, no. 1, p. 901, 2018.
View at: Publisher Site | Google Scholar
A. Fink, “Bigger data, less wisdom: the need for more inclusive collective intelligence in social service provision,” AI & Society, vol. 33, no. 1, pp. 61–70, 2018.
View at: Publisher Site | Google Scholar
N. Kolleck, A. Rieck, and M. Yemini, “Goals aligned: predictors of common goal identification in educational cross-sectoral collaboration initiatives,” Educational Management Administration & Leadership, vol. 48, no. 5, pp. 916–934, 2020.
View at: Publisher Site | Google Scholar
A. Liinamo, K. Matinheikki-Kokko, I. Gobina, and A. Villeruša, “Boosting urban health and well-being through cross-sectoral cooperation,” The European Journal of Public Health, vol. 30, no. Supplement_5, 2020.
View at: Publisher Site | Google Scholar
E. Vroblevska, I. Gobina, L. Springe, A. Bukova-Zideluna, I. Linina, and A. Villerusa, “Developing the model for cross-sectoral cooperation for promoting health and wellbeing,” SHS Web of Conferences, vol. 131, Article ID 02001, 2022.
View at: Publisher Site | Google Scholar
C. Chapman, “Place and collective impact: implications for leadership,” 2021, https://policyscotland.gla.ac.uk/place-and-collective-impact-implications-for-leadership/.
View at: Google Scholar
J. T. Croasmun and L. Ostrom, “Using likert-type scales in the social sciences,” Journal of Adult Education, vol. 40, no. 1, 2022.
View at: Google Scholar
L. South, D. Saffo, O. Vitek, C. Dunne, and M. A. Borkin, “Effective use of Likert scales in visualization evaluations: a systematic review,” Computer Graphics Forum, vol. 41, no. 3, pp. 43–55, 2022.
View at: Publisher Site | Google Scholar
K. B. Sheehan, “E-Mail survey response rates: a review,” Journal of Computer-Mediated Communication, vol. 6, no. 2, p. 1, 2006.
View at: Publisher Site | Google Scholar
S. D. P. Nayak and K. A. Narayan, “Strengths and weaknesses of online surveys,” Journal of Humanities and Social Sciences, vol. 24, no. 5, pp. 31–38, 2019.
View at: Publisher Site | Google Scholar
R. Worrall and F. O’Leary, “Towards greater collective impact: developing the collaborative capability of the local community development committees (LCDCs),” Administration, vol. 67, no. 1, pp. 73–83, 2019.
View at: Publisher Site | Google Scholar
R. Worrall and F. O’Leary, “Towards greater collective impact: building collaborative capacity in Cork city’s LCDC,” Administration, vol. 68, no. 2, pp. 37–58, 2020.
View at: Publisher Site | Google Scholar
A. Pereno and D. Eriksson, “A multi-stakeholder perspective on sustainable healthcare: from 2030 onwards,” Futures, vol. 122, Article ID 102605, 2020.
View at: Publisher Site | Google Scholar
E. Cedergren, D. Huynh, M. Kull, J. Moodie, H. R. Sigurjónsdóttir, and M. Wøien Meijer, “Public service delivery in the Nordic Region: an exercise in collaborative governance,” 2021, https://www.diva-portal.org/smash/get/diva2:1530381/FULLTEXT01.pdf.
View at: Google Scholar
D. Tell, O. Oldeide, T. Larsen, and E. Haug, “Lessons learned from an intersectoral collaboration between the public sector, NGOs, and sports clubs to meet the needs of vulnerable youths,” Societies, vol. 12, no. 1, p. 13, 2022.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2023 Tuula Takalo et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

222

Downloads

201

Citations